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one between echelons m an orgaiuzcd enterprise There is, however, a deeper 
nsychological distinction which puU reicnce into a dual role, namely, as a 
branch of the humanities and as an adjunct to technology. The tone, exam- 
ples, the orientation of the volume are eo definitely toward the latter 
of ecience that it seems adtdsable to make ati explicit acknowledgment of this 
slant. The frequently emphasised implicaUons that science is power should 
be coupled with equally important rramnders that science is also wisdom; 
i c. avoidance of self-deception rather than a giin of control. I know the two 
are closely related, hut there are also psychological distinctions between the 
two orientations which desen-c bringing out. 


I agree so thoroughly with these obeervations and so admire the 
way they are stated that I obtained Rapoport’s pennission to quote 
them here. 

This is CEsenlially a “how to do it” book rather than a “what to do 
it for” book. Furthermore it is a “how it ov^ht to be done” book 
rather than a “how it is done” book. It is a book on planning or de- 
signing the use of science in the pursuit of objectives. It leaves open 
the questions of what objectives ought to be pursued, what problems 
should he solved, and what c|UCstions should be answered. I rccogniie, 
of course, that the progress of science, in particular, and of civilisation, 
in general, may depend more on what questions are asked than on how 
they are answered. But certainly one cannot minimize the Importanco 
of answering questions and solving problems efficiently. 

Since the emphasis in this book Is on rational planning, little apaec 
is devoted to the role of hunch and intuition. TTiis is not meant to 
imply that this role is minor. To the contrary, it may be the principal 
distinguishing factor between great scientific inquiry and that which 
is mediocre or poor. But brilliant and critical insights and hunches 
cannot be planned. Their role in science is much like that of the wild- 
cat explorer lor oil. AlUiough the odds arc against him, he occasion- 
ally strikes oil and opens for exploration other regions which w’crc 
previously ignored. Nevertheless, the health and progress of the oil 
induatr>’ depend more on sysUmnlic exploration and playing of aver- 
ages than on the pursuit of the unique long shot. These extraordinary 
strikes will come no matter what wc do; we cannot plan for thorn but 
we can expect and exploit them when they come. 

In brief, this book is intended to improve the skill of the scientist 
in the conduct of inquiry. Although the philosophers have worked in 
this area and have produrwl some important overall atrategics of re- 
search, these have seldom l>cen translated into procedures that are 
u«ablc by tlie scicnti«t or into n language that can he understood by 
him. The approach to scientific incpiiry here ia much more scientific 
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than phjlosophica). It Attempts to turn science in on itself to investi- 
gate its own procedures. 

Because tliis type of effort has not been made frequently in the past, 
this book raises more <iuestions about scientific inquiry than it an- 
swers. I hnvo seldom, if ever, been aWe to find best answers, but I 
have consistently tried lo find better ones than Arc coinrnoniy acceptevl. 
My objectives will have been accomplished if I succeed in attracting 
a few scientists to the study of scientific inquirj' in its own right anti 
if A lew more find ways of improving Uieir research procedures. 

Although this book depends more on analysis tiian it does on rc- 
searejj experience, the examples througliout are drawn from my on-n 
experience. This exjvcricucc has been primarily involved with study 
of the operations of complex and purposeful man-machine systems. I 
hope that the reader will be able to find analogous examples from his 
own experience. Since my research experience has been interdisci- 
plinary in character, f hope that the- examples will emphasize the fact 
that the principles discussed arc meant to be applicable in all branches 
of science. 

^^y debts to Olliers arc many. First tiicre are those to roy teachers 
and friends, E. A. Singer, Jr. (now deceased), Thomas A. Cowan, and 
C. West Churchman, who pIow'cO the ground, planted the seeds, and 
provided the sun and the water. I can claim only to have provided 
some of the ground on which they could work. 

Secondly, I am very grateful to roy collaborators, J. Sayer Minas 
and Shiv K. Gupt.a. Let me make It dear that they cannot be held 
responsible for any of the content or the style of the book. The selec- 
tion of the material and the writing arc mine, but I have so exploited 
their thoughts, creative and critical, that an acknowledgment in a 
preface alone w'ouid not rcffect the e.xtcnt of their contribution to this 
work. 

Criticism of the manuscript from C. West Churchman, Anatol 
Ilapoport, D. V. Lindlcy, and Leon Pritsker made it possible for me 
to reduce the number of errors it contained and to improve the exposi- 
tion in many places. 

I am also indebted to roy best tcacliers, my students, who have 
struggled through several versions of this book and whose reactions 
have made many Improvements possible. I have used not only tlieir 
criticisms but afso fioiue of tftefr la partfeufar, I should like 

to thank Halim Dogrusoz, Hadoljrfi IteiniU, am! Peter C. Fishburn 
tor permission to use tlieir work and ideas. I am similarly indebted 
to my former colleague, Fred Uanssmann. 
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I am grateful to Grace White and Betty Keck for their devotion 
beyond the call of duly in the preparation of tins manuscript, and 
frequently for preparing me for work on it. 

I would like to thank the followup for permission to use material 
from their publications: The Macmillan Company; American Associa- 
tion for the Advancement of Science (Science) , Cambridge University 
Press; Holt, Rinehart and Wmstos, lac,; Methuen and Co., Ltd.; The 
American Sociological Association (The American Sociological Re- 
vim) 5 George Allen and Unwin Ltd; G. P. Putnam’s Sons; Princeton 
University Press; The Journal of Philosophy; Penguin Books Ltd.; 
The University of North Carolina Press; Eeonametrica; Macmillan and 
Co., Ltd.; Prentice-Hall, Inc.; Operationa Research Society of America 
(Operations ResearcA) ; The Royal Society of London (Proceedings of 
the Royal Society of London) ; Dover Publications Inc. ; J. B. Lippin- 
cott Company; PsycAcmjetrifca; PsycAoloyicai Review; The RAND 
Corporation; American Journal of Psychology; The Institute of Man- 
agement Sciences (ManagemenlSeienee) ; and John Wiley & Sons. 

Finally, I am particularly grateful to the University of Chicago 
Press for permission w base part of the chapter on sampling on some 
of my earlier writing on the same subject which appeared in Chapter 
IV of The Design of Serial Research (1953). 

Russell L. Ackoff 

Birmingham, England 
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1 

THE NATURE OF 
SCIENCE AND 
METHODOLOGY 


THE JtEANlNG OF ‘■SCIENCE" 

Sioee this is a book about science, vc should begin by seeking some 
common agreement as to orhat science is. Tbo e.xtensive literaturo 
addressed to the definition or characterisation of science is filled with 
inconsistent points of view and demonstrates that an adequate defi* 
nition is not easy to attain. Part of the difficulty arises from the fact 
tliat the meaning of science is not 6.xcd, but is dynamic. As science 
has evolved, so has its tneaniog. It takes on new meaning and stgnifi' 
cance with successive Ages. This evolution is not to be stopped by 
the act of defining. Althou^ wc cannot expect to attain an ultimate 
definition of science, it is desirable to reach some common under- 
standing of the concept in order (o proceed. For our purposes it is 
only necessary to agree on a few of its essential characteristics. 

First, we shall consider science as a process of inquiry; that is, ns 
a procedure for (o) answering qneslions, (6J solving problems, and 
(cj developing more effective procedures for answering questions and 
solving problems. Y'e will consider tlic distinction between que.stions 
nnd problems later. 

Science is also frequently (aken (o be a body of knowledge. iVc 
shall concentrate, hon-over, on the process which generates this knowl- 
edge rather than on the knowledge itself. 

Not all Inquiry is scientific. Here is a large class of iionsclentific 
inquirj*, inciuding wlj.nt we call "coiaaioa-sense” inquiry' TliC differ- 
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cnce between these types of inquiry is important. This difference 
must be either in subject matter or in melliod, or both. It has been 
proposed in the past that common-sense (and other kinds of non- 
scicntific) inquiry is concerned with more immediate and practical 
problems than is science. It has become increasingly clear, however, 
that science, particularly “applied science,” tloes deal with immediate 
and pressing problems. On the other band, philosophic inquiry, for 
example, is frequently directed toward problems which are neither 
immediate nor pressing. 

Some have argued that common-sense inquiry is qualitatively ori- 
ented, whereas scientific inquiry is quantitatively oriented. For ex- 
ample, John Dewey (1938, p. M) observed: 

Tlio problem of the relation of the domain of common sense to that of sci- 
ence has notoriously taken the form of opporibon of the qualitative to the non- 
qii.alilalive; largely, but not exclusively, the quantitative. 


That this distinction breaks down is clear from Herbert Dingle's 
(1053) observation that the ouUtanding scientific achievement ol the 
nineteenth century was the Uieory of cvoluUen. This theory, he notes 
(p. 6), “has nothing whatever to do with measurement. It is con- 
cerned with qualitative changes, and treaU them qualitatively." 
Even if one disagrees with Dingle’s characteriration of the theory of 
evolution, the point is not removed: an eminent historian of science is 
willing to include in science a theory that he considers to be com- 
plctcly qualitative. Furtherroore. there are obvious instances of 
fommon-scnsc inquiry which are quanlilatively oriented For ex- 
ample, a motorist who, with tlie aid of road maps, seeks the shortest 
route between two cities is engaged in a quantitative common-sense 
inquiry. 


It .cm, rfciir th.l there u a consMcnbk overlap io the questions 
and prohlcms invest, Bated reicntiOeally and nonscienlifically. It is 
al.o clear hoaever, that at the present time there are a eonsiderable 
number ol questions and proUcros (eg, ones involvina clhies) lehieh 
cannot !« truilhilly inveolieated by reienco. Tbi, has led some to 
avert lliat at least some questions and problems arc, by their very 
nature, inenpalde ol being nnsnered or solved seicntiSeally Conse- 
quently, sueli expressions n, “the limiutions ol science” are quite com- 
mon ,n du.en,sion, ol seienee Bat „ yet the limit, „I .eienee have not 
ten adequately defined, lo, ttere » hardly a type „( „r prob- 

lem to whieh science sneeeretollr nddresse. it, ell May that, at some 
■me in the pa.t, somrene did not claim n a. not snseepUble to scien- 
tillc inquio-. B 0 need not eoneern onnielve, niu, n-hciher or not sneh 
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n limit esi.sts, since it is clear tJiat, until it is we]] defined nnd gener- 
ally accepted, it cannot be used to distinguish between scientific and 
nonscicnlific inqujrj'- 

It is generally recognised that through the use of science (as con- 
trasted wit}} common sense) we arc more hkely to obtain tJjc correct 
answers to questions and belter solutions to problems. This is to 
assert not that better results are always obtained bj’ science, but that 
such results are more likely to bo obtained by its use. This follows 
from the superiority of tiie seientific process of inquirj'. This su- 
periority of scientific inquiry' derires from the fact that it is controlleii. 
A process is conirolM to the extent (hat tt is efficiently directed lou'ard 
the attainment of desired objectives. Tliomas Huxley [in Wiener 
(1053, p. 130)] matle csseotially the same obsen’ation when he wrote; 

Science is, I licUeve, nothing but trained and orgaiiueii common sense, dif- 
fering from tho latter only as a vetenn nwy differ from a raw recruit: and its 
mctlioih differ from tho«o of common sense only so far as the guardsman's cut 
and thrust differ from the msoncr in nhieh a fctvsge wields his club. 

Control of research is exercised in various degrees. Perfect control 
is an ideal which is approximated more and more closely with the ad- 
vance of science, but it is never attained. Every inquiry has some 
controlled and some uncontrolled aspects. Consequently, there are 
many gradations of inquiry* rather than the simple dichotomy; scien- 
tific and nonscicntinc. 

It should be noted that, even where scientific inquiry can do a 
''better" job than cominon-scnsc inquiry*, it is not always to be pre- 
ferred. If the cost of the inquiiy and the value of the outcome arc 
taken into account, there arc many situations in which scientific in- 
quiry is not justified. Also there are many* situations where an answer 
or solution is needed very quickly ( 04 ;., in emergencies). Here, less 
than the best answer, but an adequate one obtained "in time," is to 
be preferred to one that is belter but late. Most of the prosaic de- 
cisions that each of us makes daily about such things ns the way to 
get to work, where to eat, and what clothes to wear do not presently 
justify scientific inquiry. Science not only reaps the benefits of self- 
consciousness and control, but must also pay the associated cost. It 
should be realized, however, that much of tlie common knowledge and 
common sense that provide the basis on which today's prosaic deci- 
sions are based is itself based on the products of yesterday’s science. 

Control, though necessary, is not sufficient to distinguish between 
scientific and nonscicnlific inquiry. Sdence is also characterized by 
the goals of self-perpetuation and self-improvement. It is an on- 
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going institution which pursues an ideal: to increase without limit out 
knowledge and our ability to answer questions and solve problems. 
This imposes the requirement on scientific research that it be con- 
ducted in such a way as to increase the efficiency of future research. 
That is, research must be designed to inform and instruct us on how 
to improve the conduct of research itself. It is for this reason that 
many reports on scientific research include discussion of how the 
research ought to have been done in light of the experience gained in 
having done it the first time. In science, then, e^'ery research effort 
not only has the purpose of answering a question or solving a problem, 
. but also has the aim of testing, evaluating, and improving the research 
procedures employed. 


EXPERIMENTATION AND RESEARai 

Experimentation is sometimes taken to be identical with scientific 
research. Kot all scientific research, however, involves experimen- 
tation. Experimentation, as conceived in the nineteenth century, in- 
volved the physical manipulation of objects, events, and their proper- 
ties. Physical manipulation was taken to be identical with control. 
Tliis is reflected in the writings of F. H. Giddings (1924, p. 55) : 

In fcicatific experimenution we control everything that happens. We de- 
tcncinc when it shall occur and where. We arrange circumstances and sur- 
roundmsrs, atmospheres and lempcralurcs; possible ways of getting in and pos- 
sible ways of getting out. We take out something that has been in, or put in 
something that has been out, and see what happens. 

But our enthusiasm for experimentation may blind us to the fact that 
controlled inquiry can be conducted witliout physical manipulation. 
For example, consider astronomy. Although the situation may change 
in the future, up to now the oslrooomcr has not been able to manipu- 
late physically the objects of his study. Control can also be ob- 
tained by tlie conceptual manipulation of symbolic representations 
(models) of the phenomena under study. (We shall consider the use 
of models for this purpose in Chapter 4.) 

Though control is not synonymous with physical manipulation, 
some scientists consider it useful to distinguish between inquiries in 
which control is obtained in this way (as in a laboratory) and those 
in which it is not (os in a field survey). Tliey tend to restrict the 
use of the term expenmcntatwi to research involving physical manip- 
ulation and to employ the tcrmreieorrA to cover experimentation and 
any other type of controlled inquiry. 
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With tlie development in recent years of the techniques of dcii{fned 
experimentation (to be discussed in Chaplcr 10) it has become clear 
that physical tnanipulalion is not as necessaiy for experimentation as 
jt was once thouglit to be; it can be replaced effectively by techniques 
of classification and randomisation. Nevertheless, some physical ma- 
nipulation docs seem to be involved in most experimentation. It is 
less likely to be so involved »n the future. 


SCIENTIFIC TOOLS, TECHNIQUES, AND METHODS 

Scientific progress has been two dimensional. First, the range of 
questions and problems to which science 1ms been applied has been 
continuously extended. Second, science has continuously increased 
the efficiency svith which inquiry can be conducted. Tho products of 
scientific inquiry then are (I) a ljo<ly of information and knowledge 
which enables us better to control the environment in which we live, 
and (2) a body of procedures which enables us better to odd to this 
body of information and knowledge. 

Science both informs and instructs. The body of information gen- 
erated by eeieoce and the knowledge of how to use it are two products 
of science. As already indicated, we will not be concerned here with 
the body of information and knowledge which it has generated; that 
is, not with the specific theories, laws, and facts that have been de- 
veloped la the various physical, life, and behavioral sciences. Instead 
we will be concerned with the procedures by which science generates 
this body of knowledge, the process of inquiry. 

The procedures which characlerizc science are generally referred to 
as tods, techniques, and methods. The common inclination to use 
these tlirec terms interchangeably conceals some distinctions which are 
important to understand in discussing scientific procedures. 

By a scientific tool we mean a pliysicnl or conceptu.'il indrument 
that is used in scientific inquiry. Examples of such tools are mathe- 
matical sjTnhols, electronic computers, microscopes, tables of log- 
arithms and random numbers, therraometers, and cyclotrons. 

By a scientific technique we refer to o way of accomplishing a 
scientific objective, ti scicnUfic course of action. Techniques, there- 
fore, are ways of using scientific tools. For example, the various sam- 
pling procedures discussed in Chapter 7 arc scientific techniques which 
employ a table of random numbers, a scientific fool. The use of the 
calculus and graphic analysis are different techniques for finding the 
roinimum or maximum value of a function. 
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much like lliat of a standard of measumncnl. It performs the fol- 
lowing functions: 

(1) It provides a ha'is for detorniinuig Ihc extent to whicfi an)’ re- 
search h controlled and, hence, ecientine. In this way it provides n 
procedural goal for fcientista. The standard at any moment of time 
represents the Ijtst wo know at that lime, not for all time. Standards 
arc themselves subject to continiious tnodidention by what we learn 
in trying to use them. By making explicit our conception of the best 
research procedure we facilitate future improvements in both the 
standard and our efforts to approximate it. 

(2) Together with the appropriate theory it provides a basis for 
adjusting results obtained by use of less tlian the best known tech- 
niques 80 as belter to approximate results that would liavc been ob- 
tained by use of the best known teclifliqucs. 

(3) It makes explicit the kind of knowledge required to cfTcetively 
adjust to the ataodsrd. 


PURE AND APPUED SCIENCE 

The distinction between pure and applied science plays a central 
role in most contemporary discussions of science, ft is generally ac- 
knowledged that Uic distinction is diflicitU-— if not impossible— to 
make precise. Such a state of affairs is generally a sign tliat we are 
trying to treat qualitatively a dislincrion that is fundamentally quan- 
titative. In other words, it Is likely that pure science and applied 
science represent ranges on a scale and that a point of separation is 
diiTicult to specify. \t'hat is llie nature of Uiis underlying scale? 

Pure research is frequently characlcriied as that which is conducted 
“for its own sake.” For example, according to Nonnan Campbell 
(1952, p. I); 
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oi\Ti sake," then, means “for science’s sake." That is, the research 
is not expected to yield results wWch are immediately useful outside 
the domain of science, but Oicy arc intended to be useful within this 
domain. Hence, an operationally mcaainsful distinction between 
pure and applied research dc|H;nds on tlie nature of the intcndcil con- 
sumers of the research results. Therefore, ]jure research is research 
which docs not consider uses of its results outside the domain of sci- 
ence. It is relatively rare, honewr, tliat the results of "pure” re- 
search do not eventually become useful outside the domain of science. 
AtTiether or not they are applied depends on others. 

If one bases the distinction between pure and applied research on 
the researcher's intention, we are left widi the possibility of two re- 
search proiecls which are alike in all respects except for the inten- 
tions of the researchers involved, one pure and one applied. That is, 
the distinction is based not on a diRerencc in the land of research 
conducted, but on a property of the rwearchcr. Furthermore, since 
the researcher’s interest in application of his results outside the do- 
main of science may vary in intensity and dcETW, this dislinctioo 
seems ultimately to be quantitative rather than qualitative. This 
would explain in part the futsiness o( the distinction. 

Much of what is thought to be pure research at one time eventually 
is applied to practical problems outside the domain of science. On the 
other hand, it is clear that many of the questions to which pure 
science has addressed itself have been generated out of difBculties en- 
. countered in applied research. For example, in trj’ing to estimate 
the future demand for a specified commodity it becomes apparent 
that the cost of an error of any given magnitude depends on the 
sign (positive or negative) of the error. Commonly used estimating 
procedures Msume that such cost of error is independent of whether 
the error is the result of o>-erestimation or underestimation. There- 
fore, an inquiry may be started to develop an estimating procedure 
which can be used effectively where the cost of error depends on both 
the magnitude and the sign (i e., direction) of the error. Although 
the original inquiry may have been "applied,” the search for better 
estimating procedures would normally be thought of as "pure” re- 
search. 

A related way of distinguishing between pure and applied research 
lies in the distmclion between trying "to answer a question” and “to 
solve a problem.” This distinction will receive further attention in 
Chapter 2. For the present it is enou^ to obser\ e that an individ- 
ual has a problem if he wants fiomething he does not have has un- 
equally effective alternative ways of trying to get it, and is in’ doubt as 
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to which nllcrnativc is “best" Tlierefore, in attempting to solve 
R problem we may engsge in research to Rttain in/ormation and 
instruction in onier to decide how best to pursue the objective (s) that 
define the problem. Put another w^', in solving problems we may 
tr>’ to answer questions for the sake of better pursuing a specified 
set of objectives, If, on the other hand, our concern with a question 
does not involve use of its answer in pursuit of any specific objective, 
then we do not have a problem. 

IVJicn questions, in the sense in whieli we Itave just discussed tliem, 
are answered by scientific methods, this activity would, then, nor- 
mally be thought of as pure research. Where problems are solved 
by scientific research, if any of the objectives involved are nonscien- 
tific in character, the research is thought of as applied. Tliis does not 
mean that all problems fall into the domain of applied researcli. 
Metliodological problems, for example, belong in the domain of pure 
science, since tlie objectives involved nre scientific in character. 

Having tried to make the distinction between pure and applied re- 
search meaningful, it should again be observed that to maintain it is 
vcpj’ difficult. This is not as serious as it was once thought to be. To 
0 large extent the distinction was made in order to provide status to 
scientists who considered themselves “pure" at a time when it was 
thought that pure scientific problems required a greater knowledge and 
ability for their sotution than did applied problems. This type of 
scientific snobbery has diminished with the growth of the realisation 
that difficulty in science is not related to applicability of results. 


TYPES OF QUESTIONS AND ANStVERS 

The types of questions which science can answer, and llicir an- 
swers, can be classified in many ways. Our concern here, however, 
is with a type of clajsificfllion which sets the stage for tlie method- 
ological discussions to follow. 

Answers to questions are statements whose parts are erpressions. 
It is useful to classify statements in terms of their form and scope 
ami the form of the expressions that tliey contain. The scheme wc 
will use is the following: 

(1) Form of statements 

(a) Predicafitm — cfassification 

(b) Comparative 

(c) Functional 
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(2) Fonii of cxprenoioDs 

(a) Qualitative 

(b) Quantitative 

(3) Scope ol atatements 

(a) Particular 

(b) General 


Form of Stalcmcnls 

A statement may be represented abstractly by the following form: 

where *i, rj, •••>»« represent the eubjects or nouns of the statement, 
and F represents the relationship among them that the statement 
asserts. The eubjecla arc referred to as argument$, F is referred to as 
the jrredicaU, and n is the degree o] the predicate. Tot n = 1 (i.e., 
a predicate of degree 1), we have the subject-predicate typo of state- 
ment. For example, the slatemcnl 


has the form 


Charles u a male 


F(x}, 


where x denotes the subject “Charles,” and F denotes the (monadic) 
predicate “is a male.” For n > 1, we have a relational sentence. 
For example. 

New York is east of Chicago 

has the form 

F(r,, X2), 

where Xf and xs denote "New York" and “Chicago” and F denotes 
the predicate “is east of.” An erampic of a statement containing a 
triadic predicate (i.c., a predicate of degree 3) is 

Chicago lies between New York and Denver, 
which has the form 

Fto, Z,, T,). 

It should be noted that the statement 

Charles and Tom are males 
may be intended as an abbreviation of 
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Charles is a male and Tom is a male, 
which has the form 

F(ri) and Fir^) 

ratiier than 

F(xi,xt). 

Predication and classihc&tion 

As indicated above, a simple subject-predicate type of statement is 
one which has the form F(x); for example, 

Charles is a male. 


Such a statement attributes a property to an object, event, or state. 
A rompouJid prcdicational statement combines two or more simple 
ones. For example, 


and 


Charles ts o male IFi(r)] 
Charles is an adult (Fjfx) ] 


ean be combined into 


Charles is an adult male. 


This statement can be represented by fi(») and Fai*)- Similarly, 
the statement 

Charles and Tom are adult males 


combines two compound prcdicational statements and can be repre* 
sented by Fi(xi), Fifra), Fj{xi), and Fafijj. This symbolism makes 
e.\plicit the fact that confirmation of the statement requires four at- 
tributions. 

In order to confirm simple prcdicational statements, it is necessary 
to (a) identify tlie subject and (6) define the attributed property. 
I<lentification inrolves sjiecifying a set of properties uhich are suf- 
ficient to dillerentiale the subject from any other possible subjects. 
Hence, identification involves a compound prcdicational statement, 
[Fi(i), Fa{i}, F„(z)], whew Ft, Ft, •••, F„ ore sufficient to 
identify x, We will discuss Die fonn of scientific definitions in detail 
in Chapter 5. 

It will be noted that the statement 


is equivalent to 


Charfes ts a male 


Charles is a mcinficr of the set of males. 
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That is, cvcrj' prcdicational statement claitifics its subject. Thert- 
fore, corresponding to each (monadic) predicate, F , defined over a set, 

S, there is a subset of S eonsistins of all those raembera of S having 
the predicate F. A simple predicate applied to a set, then, creates 
two classes. If tiicrc are m predicates, 2“ classes can be constructed. 
(Classification will be discussed in detail in Chapter C.) 

Relations and comparisons 

As already indicated, a statement with a predicate of degree greater 
than 1 is called a relational statentent. In flri, Xj) a property is 
attributed to Xi and x* taken colicctis-ely. For example, in the 
statement 

C/iarlfe it the brother o/ Horace 

“is a brother of," the predicate, cannot be attributed to either sub- 
ject taken separately, as "are male" can. U will be noted Uiat in 
this statement we can revise the order of the subjects, Charles (X|) 
and Horace (xj) ; that is, 

F(Ti, Xj) implies F(ts, i,). 

'Wlierc tlus condition bolds for every' pair of subjects, the predicate is 
said to be a sj/mmeln'c relation. Such a relation does not order the 
subjects, but a relation which is not symmetric may; for example, 

Charles is younger than Horace. 

Here /’(ii, Xj) does not imply F(zs, Xi). Charles and Iforaee are 
said to be an ordered pair. 

For a relation to order more than tnv subjects it must be trarm'lii-e, 
in addition to not being symmetric. A (dyadic) predicate is said to 
be transitive if and only if, for any triplet of arguments, x, y, and s, 
F(r, V) and F(y, z) togetter imply F(x, r). A comparative state- 
ment b any statement the prinripal predicate of which is an ordering 
relation. For example, the predicate "ia less than” defined over the 
real numbers provides an ordering of the real numbers. 

Ordering relations are of two ^rpes, quasi and strict, dependmg upon 
whether the relation is refierive or irre^cxiic. A (dyadic) relation 
F defined over a set S is said to be reflexive if and only if F(r, x) is 
true for every x in S. It is said to be ineSexive if and only if F(x, x) 
is false for every x in 5. 

Examples of quasi-ordering relations are "less than or equal to” in 
the domain of real numbers, "fa at least as tall as” in the domain of 
human beings, and “implies" in the domain of statements. Examples 
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of strict ordering relations arc “is less than” in the domain of real 
numbore, “is the ancestor oP' in the domain of human beings, and “is 
a proper subset of” in the domain of sets. 

There are many different types of ordering relations, some of which 
will be discussed in detail in Chapter 6. 

Functions 

A particularly important class of relational statements consists of 
ones involving a funclionat relation. In a statement of the form 
F(ti, Xi •••, X,), where n > 1, if when F and all but one of the 1*8 are 
specified, the value of the remaining * is completely determined, 
then F is a strong functional relation. For example, consider the 
(dyadic) statement 

Glorin is the spouse o/ Charles, 

which can be represented ns F(xi, ij). Once F is specified ns “is the 
spouse of” and either Xi or t; is specified (Gloria or Charles), then 
the value of the other is completely delcnnincd. This statement may 
be rewritten as cither 

or 

xj - /j(xt), 

which are in the xisual functional notation. 

Consider the triadic predicate F defined over the real numbers such 
that F(,Xi, Tj, Xt) means "Xi is the sum of i2 and Xj.” Such a predicate 
yields a function for all its arguments, and we may write 


xi =/i(xj,Xa) 

Xi “ /j(*ii Xa) 

*8 =• h(X2, Xi). 

In this case, 

/j(X3,X3) = Xs + Xj 
/ifXl, Xj) = Xj — Xj 
/aCxi.xa) = xj — zj. 


Note tlie important properly of statements involving strong func- 
tional relations; if tlie value of any (independent) argument inside the 
functional bracket is changed, the value of the (dependent) argu- 
ment on the Jeft side of the equation must be changed. 



14 SCIENTIFIC METHOD 

Now let US consider a u-eofe functional relation; for example, the 
dyadic predicate "is the fa^er of" in the domain of human beings. 
Fir,, ij) means "ri is the father of For any given value of xa, 
there is only one value of Xi such tiiat F(ri, Xi) is true. In this case, 
however, specifying Xi does not detenoine xi, since Xj may be the 
father of several persons. In general, a predicate is a weak functional 
relation for its Jcth argument if and only if, (a) when the values of all 
arguments except the ith are fixed, precisely one value for the kth 
argument Is determined, and (b) a change in an x other than Xs may 
not necessitate a change in t». For example, in the statement 

F. D. R. u'os the father of James Roosevelt 

it "F. D. R," U changed, “James Roosevelt” must be also; but, if 
"James Roosevelt" is changed, “F. D. R." need not be (if one of his 
other offspring is substituted for James). In the earlier example in 
which F denotes "is the spouse of," both X| and Xs were su/licKnt to 
completely determine the other. In this example, Xs is sufficient 
(relative to the predicate “waa the father of”) to determine Xi, but xi 
is not sufficient to determine x.. However, Xi is sufficient to specify a 
class of subjects any one of which substituted for makes the state* 
ment true; therefore. Xi bounds the values of Xs. 

When we examine the type of statements yielded by scientific re- 
search which take the form 


*l * /(Xz, Xl, • • •) 

we observe three different types which are characterized by the 
property of the function. Consider first the familiar law of freely 
falling bodies: 

* = 

in which « is the distance traveled, g is the gravitational constant, and 
i is the time from release. We note that (for nonnegative s, g, and t) 

where/|(g,0 = 

ff = /zC®, 0, where /g(#, () = 2</{* 
t = /*(«, jr), where /j(«, g) = V 2*/g. 

Clearly, the functional relation involved in this law is strong, smcc 
the value of each argument is completely determined by the other 
two. 
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Now consider a statement of the Jona 


•Tl -S(Xi,Xt, •••, Ti), 

wlicre X 2 , X 3 , • • •, iii U a subset of a set of arguments which is sulScicnt 
to completely ilclcrminc the value of X|. The subset, then, only par- 
tially determines (i.e., bounds) the value ot Ji. For example, suppose 
that in fact 

ii =* ij + X3, 


that Xs and Xa are independent, and that Xj can assume three different 
values: —1, 0, and 1. Suppose further that we do not know about Xj 
but we do know that the value of Xi depends on the value of x< and 
something else. Then, from observation we could determine that 
cither 

(a) xi = i 2 - 1 


ii) X( - xj 

or 

(e) Xi-X2 + 1. 


Suppose also that the probabilities of observing each were p(a) « 
0.25, p(b) es 0.25, and p(e) » OiiO. We could now compute 
the expected value of Xi: 

E(x,) - 0.2o(xa - 1) + 0.23(xj) + O.SOfxj -f- 1) 

»» 0.25xj — 0.25 + 0.2.'»X5 + O.SOxg + 0.50 
= ij 4- 0.25. 

Now, although the expected value of X|, EfXi), is eompletcJj' deter- 
mined the value of X| is not. We know that a change in xj is not suf- 
ficient to result in a ch.nnge in Xj, since a change in zg may compensate 
for it. But we do know tliat knowledge of the value of xs is necess.ary 
for determining tJ>e value of x,. Then x* is not a deterministic cause 
of Xi, but it is a probahiliitic cause or producer of Xi. 

Suppose that we do not know whether the value of Xt depends on 
tlic value of Xj; that is, we know of no necessary connection between 
xi and Xjf, but we have observed that xi tends to increase as x- does. 
Once again we may express Xj as a function of is, but this is a pseudo 
function, since Xg is not suficient for, and « c do not know that is neces- 
sary for, determining the value of Xi. tVe cannot say that xj is either 
the cause or the producer of Xj, but wc may be able to say that they 
are correlated. 
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We have noted three types of functions: 

(1) Cause-effect (detcrmjHwtic causality;. 

(2) Producer-product ( probabilistic causality). 

(3) Correlation. 

Unfortunately, causc-cffect has frequently Iwcn treated ambiguously 
in science, and hcncc the important dutinction between cause-effect 
and producer-product rclationsiiips is ecldoro made, We will consider 
these relations in detail in Chapter 10, but m the meantime they may 
be distinguished as follows. 

When one phenomenon, X, is said to cause anotiier, V, several dif- 
ferent things may be meant; 

(1) A' is necessary and sufficient for Y. 

(2) X is necessary but nob sufficient for P. 

(8) X is not known to be either necessary or sufficient for V, but 
they tend to be present or absent togetlier. 

The first of these is deterministic causality, the second is probahih'sftc 
or nondeterministic, and the third is eorrefatfon and may not involve 
causality at all, 

Xow let us make the meaning of Uiese three types of function more 
precise. Let Ei and Cj represent the environments of X and P, re- 
spectively. Tliey may or may not be the same. Let the syunbol 
represent “is always followed by”; and “X in E" and ‘‘P in Es" repre- 
sent X and y in their respective environments. 

Suppose that a phenomenon of type X in an environment of type Ei 
is always followed by a phenomenon of type Y in an environment of 
type Eo\ that is, 

A' in X, -* r in X,. 


Then X in Ex is suffinent for V in E-: whenever X occurs in Ei, 1’ 
later occurs in Ej. Consider the role of A*, given its environment 
Let X' and V' represent the nonoccurrcncc of phenomena of type X 
and 1’. If both 


and 


X in £, — * Y in Ej 
X' in r, -» Y’ in Ei 


are true, then (given Et) X is both necessary and sufficient for Y, 
since Y occurs only if X does, and Y always occurs if X does. Within 
the environment Ex, X completely determines the occurrence of Y in 
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Es and hcncc can be sanl to be tlie deterministic cause of 1*. For ex- 
ample, if we can define nn en%’ironment, Ei, within which striking a 
bell, X, h always followed by a ringing of the bell, V, then in the 
environment Ei striking the hell U Die tlctermini''tic cause of its 
ringing. 

Xow Eiipjiopc that two phenomena, Xt and A'a, arc jointly necessary 
and siilficicnt in Ei tor the eubsequent occurrence of F in Ej; that is, 

A'l and A'j in Et —* I*xu Ej 
A'l* anil A*j in Ei — * }" in Ej 
A'l and AV m E| — ♦ Y‘ in Ej. 

Phenomena of the type A'l arc ncec5«ary but not sufficient for J’. If 
A'l occurs, Y occurs or not, depending on whether A'* occurs. Hence, 
the probability that 1' wiif occur given A'l in Ej depends on the prob- 
ability that A'j will occur in Et. Consequently, A’l (and Xt) ore 
probabilistic (nondcterministic) causes of V. We will call A'j and A's 
pTotiuten of 1’, and V Uic pro</M<t of A't and A'j. If, in addition to 
the three conditions given above, tlie following two al.«o hold: 

A'l and A*, in E,' — 1" in E* 

A'l' and A’j' in E% -* )'' in Ej, 

then the environment Ei is abo necessary but not sufficient for 1', and 
it too Is a producer (a coprwluccr) of Y. 

In genera], anything which is necessary but not sufficient for tlio 
subsequent occujTcnce of another thing is the producer of the second 
thing. For example, a seed does or does not produce a plant in some 
plots of grounil, depending on wlicllicr there is enough sun and water; 
or n Jjcll in a jar rings if struct, providing enough air is present. 

Now suppose that the following conditions hold: 

.Yi and AV in Ej — * F in Ej 
A’j' and Xj in E| -♦ F in Ej 
A'l' and Xj' in Ei — • Y* in Ej. 

Tlien, given Ei, A'l and A'j are separately sufficient for 1', but neither 
is necessary. Neither A'l nor A'a can be said to be the cause of F, 
since a cause (deterministje or probabilistic) must be necessary for 
its effect. In all likelihood, however, in such cases Xi and A'j have a 
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common property winch is nwe'-ary for Y ami lienee is a cause of it. 
For example, a person may be allergic to lioth bread and cake. F>itber 
one leads to an allergic rcacUon. Tlic person may be reacting to the 
wheat flour which they have in common. The flour, not the bread or 
cake, produces the reaction. In speaking loosely we may refer to the 
bread or cake as producing the reaction, but in science such inexact- 
ness is to be avoided if possible. 

Finally, there is the situation in which X in Ei may sometimes 
be followed by Y, and sometimes not. Furthermore, X' in Ei may 
sometimes be followed by }’ and eomctimes not. But suppose that 1’ 
is more likely than not to occur when X has preceded it. Then we 
say that the occurrence of A' and V is positively correlated. 

Conader, for example, a person who usually brushes his teeth once 
a day, just before going to sleep at night. Brushing his teeth is 
neither necessary nor sufficient for his going to sleep and hence is 
neither the cause nor the producer of his retiring for the niglit. And 
yet the two events usually occur together. To take another example, 
in one large city U was discovered that people who live in neighbor- 
hoods in which there is a hea %7 soot-fall are more likely to get tuber- 
culosis than people who live in neighborhoods with less soot-fall. Yet 
medical research has shown that soot-fall is neither nceessary nor 
sufBcient for the occurrence of tuberculosis, lienee, the valuH of two 
variables may lend to change together, and yet the variables may not 
be causally connected. Such variables are said to be correlated. 

The knowledge that two things tend or do not tend to change to- 
gether can, nevertheless, be very useful. For example, when we see 
the person in the above illustration brush his teeth at night, we can 
predict with some assurance that he is about to retire. That is, we 
can Use our knowledge of the value of one variable to predict the 
value of another. 

Correlation analysis enables us to measure the tendency of vari- 
ables to change or not to change their values togelJier. We shall dis- 
cuss the technical aspects of such analysis in Chapter 10 But it is 
important to post a warning here — a warning which shall be posted 
again, since failure to heed it leads to one of the most prcv'alcnt 
errors in the social sciences. To establish that two tilings tend to 
change or occur together is not to establish Uiat they are related di- 
rectly or even indirectly by a producer-product or cause-effect rela- 
tionship. We cannot infer production or causation from correlation 
alone. On the basis of the coradation between soot-fall and tubercu- 
losis mentioned above, one researcher concluded that soot-fall was a 
producer of tuberculosis. But subsequent research showed Uiat dietary 
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deficiencies nrc in fact protluem of tuberculosis. Furtlior, dietary de- 
ficiencies are likely to occur most frequently among low-income 
groups. liow-incomc groups arc likely to live in low-rent <listrict3. 
Districts have low rent, among oUier things, because of hcn%y soot- 
fall. Thus soot-fall and Idberculosis arc accidentally, rmt e«cntially, 
connected. 


Form of Expressions in Statements: Quality and Quantity 
Comp.are the following two slntcinenU: 

John is henvp 
and 

John treighs I JO poumU. 


Iloth appear to be tiniplc pre(licaUon.al statements of the form FfrO, 
where X| denotes "is heaAy" jo the former and "weighs 150 pounds" in 
the latter. Tlic obvious difTcrence l>ctwecn these two statements is 
that the second contains a numbtr. What is not so obnous is that, 
because the second statement contains a numiwr in what Api>cars to 
l>e its predicate, it should be represented as a functional statement of 
the form F(X|, xt), where F denotes "is equal to,” xi denote.? "John's 
weight,” andxj denotes "ISO pounds." This is a vtak function, since 
specification of F and Xi completely determines Xj, hut F and xj do 
not dctcrniine xi. 

A transformation similar to changing 


into 


Joh/i xceighs J60 pounds 


John’s trer^ht is tqunl to toO pounds 


cannot be performed on 

John is heavp. 

We can tr.ansfonn this statement into 

John’s ireight ts ffreatcr than U’ pounds 


or 

John’s weight is greater than Wi pounds and less than Wi pounds. 
There is, liowevcr, no reasonable transformation of "John is iicavy" 
into a statement containing the relationship of strict equality. 
Xumbers may be u-ed in statements for a variety of purposes: 
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(a) To identify (or name) the subject, for example, 

This is cab number 4SS. 

(b) To identify the class in which the subject is placed; for ex- 
ample, 

He is a registered voter in Ike Fourteenth 'Ward. 

(c) To identify the number of subjects in a class; for example, 

There are 5S2Q voters in the Fourteenth Ward. 

(d) To identify the rank order of a subject in a class; for example, 

The Fourteenth U’ord is the third largest in the city. 

(e) To identify the number of units on a scale which corresponds 
to the subject's property; for example, 

John weighs ISO pounds. 

Each of these statements can be put into a weak ftmetioD form, but 
only the last is an example of what we normally think of as measure- 
ment. We will consider all these types of number assignment in 
Chapter 6, but here some observations on prepc^'es wliich are treated 
qualitatively and ones which are measured (1.^ treated quaniita" 
tlvely) are in order. 

It is important to observe that qualitative predication may involve 
quantification. For example, in order to determine the color of an 
object it may be necessary to measure the wave length of light re- 
flected from it under certain specified conditions. 

Any property which can be quantified can also be treated quali- 
tatively. A quality can be thought of as a range along a scale in 
terms of which the properly can be measured. For example, a person 
can be said to be "tall" if he is over 5 feel 10 inches, "medium’’ if 
he is between 5 feel 6 inches and 5 feet 10 inches, and “short” if he is 
under 5 feet 6 inches. 

It is also true that any qualified property is potentially capable of 
being expressed quantitatively in terms gf such a range along a scale. 
IVe may never be able to translate all qualities to such measures, but, 
as science progresses, it converts more and more qualities into equiva- 
lent quantitative expressions. But this is not a one-sided development. 
As science develops more measures, it also requires new kinds of quali- 
tative judgments. For example, hei^t can be measured as a vertical 
distance; but to do so requires our ability to determine verticality. 
We can convert verticality into a measure of the angle between a 
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«traig?jt Jlnc one! a radius projected from tlic earth's center of gravity. 
This requires our nbiJity to determine straigiitnoss, and so on. Quanti- 
fication at any stage depends on qualification. What is qualified at 
one stage may be quantified at another, but at any stage some quali- 
tative judgments are required. Consequently, progress in science not 
only is a function of an increased capacity to quantify efficiently (i.c., 
to measure) but also depends on an increased capacity to qualify 
efficiently. 

The Scope of Statements: Tlic particular and the Ccncra] 

One statement can be said to be more general than another if it 
impffca and i« not tmph’sd by the otlicr; that is, if the truth of the 
second necessarily -follows from the truth of the first and not con- 
versely. Scientific statements arc about things under certain condi- 
tions. The larger the class of things to uhicli reference is made, and 
the raorc inclusive the set of condiUons, the more general b the state- 
ment, For example, the stalemcDt 

All A'’s have the property 1* itnder conditions (Cj, Ca, • • •, C«) 
is more general tho'^Uie statement 

This X has the property 1' under eondftmnj (Cj, Ca, •••, C,). 

Each statement can also be further gcocraJired by enlarging the range 
of environments in which it bolds. 

The less general a st-atcnient, tfie more /act-ltke it fs; the more 
general a statement, the more lav-like it is. Hence, facts and laws 
represent ranges along the scale of generality. There is no well- 
ilefined point of separation between these ranges. 

General statements are of two types. The first type refers to a 
class of events or conditions each instance of whicii has been observed. 
Tlie second tj-pe refers to classes of things and/or conditions some of 
which have not been observed and all of which can never have been 
observed. If, lor example, we observe that each of four pots of water 
boils at 100® C and make a statement to tins effect, then this state- 
ment is of the first type. If we infer from these observations that 
the boiling point of water at standard atmospheric pressure is 100® C, 
we have made a statement of flic seconcf type. Tlse tern iaccr is 
normally restricted to general inferential statements of the second 
typo whicii in addition assert a causal (detenninistic or probabilistic) 
relationship. 
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A theory is a still (urtlict prncralitalion. Wolf (I92S, pp. 126-127) 
pointed this out as follows: 

. . . Newton’s theory of En%jWtion (fspccully in its original eauMl fcn*c) u 
an explanation of Kepler's three laws and of Gahters Iaw of Falling Bodies, 
the Kinetic Tlieory of Gases is an expbnation of Boyle’s, Avogadro’s, and Gay- 
Lussac’s Laws (and of the feiwnlc laws which they sunimanxe); and the Un- 
diUatory ’Theory of Lght cxpliina Snell’s Law of llrfraction (and the laws of 
which this u a summary) by reducing the bendmg of a ray of light, as it passes 
from one medium to another of difiercnt density, to differences in the velocities 
of light in the two media. 

There is a tendency to distinguish such a more comprchennvc Law from th't 
less comprehensive laws, which it explain*, by railing it a TArory. 

Perhaps the relationship betneen theory, Iftw, and fact Is Best 
grasped in the context of a deductive system. In a deductive system 
there arc (1) a set of undefined and defined concepts, (2) n set of 
assumptions (axioms and postulates, or formation and transformation 
rules), (3) a set of deduced theorems, and (4) instances of the the- 
orems. The assumptions eonsliliile the theory, the theorems consti- 
tute laws, and Uie instances of Uic theorems are the facts. In the 
construction of scientific theories the objective is to construct )ust 
such a deductive sy’stem. 

A theory, however, is mere than a Beneraliiatien from which laws 
can be deduced because, as Campbell (ld32, pp. 82-$3) observed, 

. . . there are an Indefinite number of "theories” from which the laws could be 
deduced; it is a mere logical exercise to fin<i one set of propositions from 
which another set will follow. . . . For Instance, that the two propositions 
(1) that the pressure of gas increases os the temperature inere.-iscs (2) that 
it increases as the volume decreases, can be deiluced from a single proposition 
that the pressure increases with increase of temperature and decrease of 
volume. But of course the s'tngle proposition does not explain the two others; 
it merely states them in other words. 

A theory must, according to Campbell (1^52, p. B9), aatisly two addi- 
tional conditions: 

... it must explain those laws in the sense of introducing ideas which are more 
familiar or, in some other way, more acceptable than those of the lai^; [and] 
it must predict new laws and these laws must turn out to be true. 

In the assertion of three levels of generality in facts, laws, and 
theories it is frequently implied Uiat because of differences in gener- 
ality facts are simplest to confirm, laws more difficult, and theories 
most difficult. Now this is true in one sense: in the sense that at 
present the methodology of confinning facts is much more highly de- 
veloped than is the corresponding methodology for dealing with laws 
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anti ilieorics. For example, Ciiurehman (iflCI, pp. 76-77) ottserves 
that 

Thfrp are an infinite numl>rr of hypotlkscs conljiiiicd in . . . [a] theory. 
tSTiich ones should be tcsitxl? 

One miaht appeal to common and a>tcrt that the tests should be 
‘Vpread out” equitably over the inlen'at. 

But what do "equitably" and "the interval'' mean? Churchman (1961, 
p. 77) points out tliat this is not easy to answer. 

In practice, ne need a theory of ibeory-tesUog . . , m<wt »cj«iiisu recog- 
nize th.at the mere nccutnul-slion of mcasurrmenU that “eonfinii" a theory do 
not add to one’s confidence in the theory, for they nuiy merely over-test one 
or more hypotheses, and under-test tlie rest Thus, on a vague level, one feels 
that a theory ought to be tested under "widefy difierent” circmustauecs, Init 
tills prescription is difficult to define. 

There is another sense in which theories arc held to be more difficult 
to confirm than fact*. It is maintaintnl by some that the eonfinnation 
of facts does not require use of tlieory but that confirmation of the- 
ories docs entail confirmation of fact. This is not true. For example, 
Churchman (1961, p. SI) notes: 

... the praeUeo of science shows that sucii a propoation [stalenient] aa "An 
observed object X weighs k pounds" or "An observed object A' is blue" are, 
methodologically interpreted, elliptical for "If X is obniervcd under any of the 
conditions C, belonging to the class of conditions C, by any obserter 0, be- 
longing to the class of observers 0, tho reconled observations wiU all liclong 
to a ehss of propositions P." The confirmatioB of this proposition heJongs to 
the last type — the theories. By a similar argument, all other propositions of 
science are — methodologically interpreted— theories. 

In Cli.apfers 5 through 10 we will explore what is involved in "proving” 
or "finding” facts. It will be clear in this discussion that the con- 
cept of confirmation of even “aimple” facts by direct and immediate 
cjb‘mstion is a methoilologjrfll myth inherited from tho past. 

It 13 maintained by some that, although laws and theories are more 
difficult to confirm than facts, they are less difficult — or at least no 
more difficult — to disconfirro Ihan facts. In principle, a Jaw or theory 
can be discontinued by just one contradictory fact. But in practice 
the fact which appears to contradict ttie law or theory is itself always 
subject to doubt. Consequently, Uicre have been many historical in- 
stances where facts which appear to contradict laws or theories have 
been rejected in order to maintain a law or theory in which the scien- 
tist had more confidence than he did in the fact. Furthermore, as 
Churchman (1961) and many before him have shown, the confirma- 
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tion of a fact itself presupposes a theory and hence the validity of fl 
fact always depends on the validation of the assumed theory. For 
example, it was once assumed (and still is by some) that, if facts and 
theory do not agree, then the theory must be false. As the history 
of science shows, however, the scientkt is just as likely (if not more 
so) to question tlie validity of the facts as he is to question the valid- 
ity of the theory. 


TYPES OF PROBLEMS 

Research that is directed toward the solution of problems can also 
be divided into two major classes: ei'afuotit'e and developmental. An 
evaluative problem, h one in which the alternative courses of action 
are complexly specified in advance and the solution consists of select- 
ing the "best" of these. A developmental problem involves the search 
for (and perhaps construction or synthesis of) instniments which 
yield a course of action that is belter than any available at the time. 
For example, the selection of the best of two or more existings drugs 
for curing a specified ailment is an evaluative problem. Developing a 
better drug than any available is a developmental problem. In dls- 
cussbg the phases of research we shall consider each of the types of 
research that have been identified and explore their methodological 
differences and sioularitics. But the basis of these comparisons will 
be laid throughout by a detailed consideration of evaluative problem 
solving because here, as we shall try to show, we are presently capable 
of attaining our highest degree of methodological self-consciousness 
and sophistication. 

As we shall see in some detail, applied research has the advantage 
of being able to formulate criteria of its own efficiency in terras of 
the objectives for which the problem is being investigated. Because 
of its lack of specific objectives, pure research cannot formulate such 
criteria as explicitly. Consequently, pure research makes many im- 
plicit assumptions about the conditions under which its results will he 
applied. In applied research these assumptions are frequently found 
to be unrealistic. To elaborate a prenous example, in pure research 
the seriousness of various errors can seldom be measured. Hence esti- 
mates are made in a way that is "best" only if the seriousness of an 
error is independent o! its agn; that is, if the seriousness of an error 
of +x is equivalent to the seriousness of an error of — i. In applied 
problems, however, there are few eases in which this condition holds. 
Hence different estimation procedures are required in applied ecience, 
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and acriovs questions about tJie eatimating procedure 0 / pure science 
are raised. (This problem will be discussed in detail in Chapter 8.) 

It will be observed, as wo go into detailed discus.sion of ficienlific 
methofl, that in genera! wo bare tte ofijiortunity ol making research- 
<!c&igM (mctlioi1ologie.nl) dccinions more self consciously in applied 
science than in pure science. Tins fact w not generally appreciated; 
to Uie contraij', it « commonly Ijelievod tiiat pure research tends to 
be methodologically superior lo applinl research. Hence the general 
approach of this book may be contrary to the intuition and beliefs of 
many. 

U’c shall first discuss the melhodologicnl aspects of each phase of 
research in an ajipHcti context, and then consider what we can leam 
from Uiis presentation that can be used in the pure-researcb context. 

THE PHASES OF HESEAHCII 

The phases of research have traditionally been identified as 

(1) observation, 

(2) generalization, 

(3) experimentation, 

or as some variation of these. For example, D'Abro (1051, p. 3) lists 
the stages of Uio scientific method as 

(o) the observational stage, 

(6) the experimental stage, 

(c) the tlieorctical and mathematical stage (in physics). 

He then comments (p. 3) on tlicir sequence as follows; 

The order in which tlicse stages have been listed is the order in which they 
arise in the study of any group 0 / physical pbenomraa. It is also tlic ebrono- 
logical order in which they were discovered. 

One could lake issue wiUi tbe assertion concerning chronology, for 
example, on the grounds that atomic theory nas app.xrentJy formu- 
lated in qualitative form by Democritus without his engaging in either 
of the first two stages. Tliis issue, however, is not as important as 
tlie one arising from tbe ImpUcation tbat the order of tbe steps is 
fixed. The positha we shs)} take here ia that s!) sisges may and 
usually do go on simultaneously. Some previous or new theory may 
suggest and direct new observations. The empirical tradition asserts 
the primacy of observation in science, but more sophisticated philo- 
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sophical analysis has shown that observation always presupposes a 
criterion of relevance, and this criterion, in turn, always involves some 
theorj'. This is not to assert the primacy of tlieory. To the contrary', 
some modern philosophies of science, such as pragmatism and ex- 
perimentalism, assert tlic “cyclic" and inlenlepcndent characteristic 
of these stages of scientific mctliod* 

The various tlirec-stagc breakdowns of the rcscarcli process have 
been made primarily by scientists preoccupied with pure research. 
From the point of view of applied research a finer breakdown is nec- 
essary. There are almost as many listings of applied-research phases 
as there are persons nho l»ave tried to provide such a classification. 
Since most such listings are essentially equivalent, it docs not make 
much difference which one is used as long as its meaning is made 
clear. Here we shall discuss research in six phases: 

(1) Formulating the problem. 

(3) Constructing the mode). 

(3) Testing the model. 

(4) Deriving a solution from the model. 

(5) Testing and controlling the solution, 

(6) Implementing the solution, 

This hook is organized around a discussion of each of these phases of 
lesearch. These phases arc not discrete stages each of which is com- 
pleted before the next is begun. In general all phases go on simul- 
taneously and are completed together. They frequently are begun, 
however, in the order in which they are listed. 

The research process is usually cyclic. For example, if, when test- 
ing a model, it is found to be deficient, the formulation of the problem 
and the model may be re-examined and modified. This leads to new 
testing and, in some cases, to further reirision in the formulation of 
the problem and the model. 

It will be observed that formulating the (applied-research) prob- 
lem has much in common with tiie “observational” phase of pure re- 
search; "generalization" with “model construction” and "derivation of 
the solution,” and “experimentation" with “testing the model and so- 
lution.” The control phase of applied research corresponds with ef- 
forts to further generalize the results of pure research, and implemen- 
tation corresponds with efforts to use the results of one piece of pure 

•For detailed discussion of Ibe pnmacjr of observation or theory see Church- 
man (1961), Churchman and Aekoff (1950). Dewey (1929 and 1938), and Singer 
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research in another. As indicatnl, these simiKarities anti difference 
will be explored in some detail. 


Tiin pinLosopiiY OP sarscE 

Before about a century ago most of wliat we today call science was 
c.alled rmtural philosojihij. ritilosnpluc imuiiry and ecientiGc inquiry 
were not differontiatwl from each other, at least popularly, until about 
the luidillc of ^le ninetemth cen tury. In the days when nil scientists 
were philo<ojihera nnd nio^l phihixophers were scientists a great deal 
of attention wjis given to the way in which knowledge was acquired. 
Inqiiiry into this proceduns-which was more phiIo 5 opluc.nlly than 
scientifically oriented — was alternately called epittemology and the 
theor y 0/ knou' Udge. ~ 

Ihe sepnrnlion of science from philosophy there camo an in* 
creased awareness of the fuperinritv of th e methods nnd techniques 
of science for acquiring knowIe«lge. Conf^pitritly, those who were 
concerned with the Iheorj* of knowlinlg o turned more and more to the 
analysis of scientific melho<l. Since tlds inquiry was iuclf largely 
speculative in character, it renmined philosophic and tamo to be 
known «.s the phiiosophy 0} adenee.* Scjcntisls as neU na pfailoso* 
pliers engage in the philosophy of science, which is one of the few 
rem.ainlng grounds on wh ich they mee t. But even this ground has 
been shrinking as the study of scientific method itself has become less 
and teas gpeculat ivc nnd more and more scientific. Today the breach 
between the philosophy of science nnd the science of methodology is 
wide enough to make it diflicult for all but a fcw'lo straddte 

This breach is unfortunate. Although most contemporary profes- 
sional philosophers liavc little ktiowlc<lge of present-day science, they 
do know the history of epistemology and the theory of knowledge and 
arc interested in molliodolojpcal problems. Scientists, on tlic other 
hand, generally know liille of Ibis history nnil arc inclined to take 

* Although the study of sriraliCe Method has been a major part of the phi- 
losophy of srimee, St has by no meaai been the only part. Tliis branch of 
philosophy has hid at b'Cst three other types of inUrrest in science: 

<0 Cbnccpfnal analyst*; the to define concept* or problem areas in 

such >i way as to make them sin^eptible to Kietttifie study. 

(2) Eiaminntion of assumption* concetiunit the nalura of reahty which ”un- 
uhtfle'" science. 

(3) Synthesis; Hie atlcmpl to fuse tho fimlinra td the various branches of sci- 
ences into ono coMislent view of reahty, a ITcftimschautinp. 
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for granted their own methods of acquiring knowledge. As a conse- 
quence, methodology has developed slowly, and the practices of scien- 
tists often fail to incorporate the r^lto that earlier methodological 
inquiries have produced.* 


CONCLUSION 

Jlethodology can be considered to be a special type of problem 
solving, one in which the problems to be solved are research problems. 
We will see later that any problem situation, and hence research- 
problem eiluationa, can be represented by the following equation: 

V=J{X,,Y,), 

where r = the measure of performance or accomplishment that we seek 
to maximize or minimize. 

Xi =■ the aspects of the situation we can control; the "decision” or 
"choice" or ‘'control” variables. 

5’/ - the aspects of the situation (environment of the problem) 
over which we have no control. 

Then solving a problem consists of finding those values of the decision 
variables, X, [expressed as a function, p(y/)], which maximize (or 
minimize) V. Theoretically it is posable to formulate problems in 
research design in this way and to find ‘'optimizing” solutions. The 
attainment of such optima is the objective of methodology. At the 
present time we can formulate only a few research problems in this 
way, but this achievement is the product of just a few years of such 
study of research procedures. Even where we cannot find such 
optima, liowcvcr, we can Icam a great deal about the relative effective- 
ness of different research procedures by attempting to formulate them 
in this way. Such an effort is the most cHicient way we have of pre- 
cisely defining problems in methodology and of directing scientific 
research to their solution. 

Tills book will raise many more questions than it will answer. In 
methodolog)', as in tlie rest of science, the solution of one problem 
raises several new ones. But, in general, progress in methodology and 
science is as much a matter of moving from question to question as 
it h o! iTioYing !roni saswer ta sastevr. 

•There have been many itodtes of the derrlopment of epistemology nnrl phi- 
losophy of science. See, for ezampte, ChuKhman and Ackoll (1950). 
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2 

THE MEANING OF 
"OPTIMAL SOLUTIONS 
TO PROBLEMS” 


HIE NATURE OF PROBLEMS 

The minimal necessary and sufficient conditions for the existence of 
a problem are as follows: 

(1) An tndiindual who has the problem: the decision mazier. 

(2) An outcome that is desired by the decision maker (i.e., an ob- 
jective). Witliout the desire to obtain an as-yet-unattained outcome 
there can be no problem. Therefore, in the state of Nirvana — in 
which all desire is supposed to be suspended — there can be no prol)- 
lem. An objective, then, is an outcome which has positive value for 
the decision maker. Every problem situation must involve at least 
two possible outcomes (O and not-0), but only one outcome may 
have a positive value. 

(3) At least firo unequally efficient courses of action which have 
some chance of yielding the desired objective. There must be a “real” 
difference between tlie choices available to the decision maker. If he 
has a choice between equally efficient or completely inefficient alterna- 
tives, lie may think he has a problem, but the problem has only sub- 
jective (not objective) reality. 

(4) A state of doubt in the decision maker as to which choice is 
“best.” There must be a question in the mind of the decision maker 
ns to which choice to make. 

(5) An cntironmcTtl or context of the problem. The environment 

30 
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consists of all factors w}>icli can elTcet Ok? outcome and which are not 
under the decision maker’s control. 

Problems, of course, may be considerably more complex than the 
minimal one described. Complications may rcfsult from such condi- 
tions as the following; 

(o) A group of individuals, ratlmr than one, must make the de- 
cision. 

(6) The decision makerfs) make the decision but others carry it 
out. Such is the case when managers of industrial, military, or gov- 
ernmental activities make policies or plans which others must follow. 

(c) The decision may have an effect on others who may react to 
it in such a way as to affect its efficiency. Reactions which decrease 
the efficiency of the original action are called ceninteratUons. Those 
in conflict or competition, for example, tend to counteract each oth- 
er’s action. 

(d) ^lore than one objective may be involved in a problem, and 
the objectives may not bo consistent. For example, a person may 
want "to save money” and still “have a good time” in circumstances 
where fun is not (o he had without spending. Furthermore, tlic ob- 
jectives may change with time. 

(s) Tlie number of possible courses of action may be very large; 
in fact, infinite. 

To solve a problem, w’hether simple or complex, is to make the best 
choice from among the available courses of action. 

In order to maximize our chances of attaining or approximating the 
best or optima] solution to a problem, it is apparent tliat we must 
understand what the “beat" or “optimal" solution to a problem is. In 
determining what such a solution is wc are concerned with the choice^ 
a decision maker should mate, not necessarily with those he normally 
makes. 

It is not at ail obvious nhat is meant by the "best” solution to u 
problem. A final definition of "best" in this context has not yet been 
attained, and it is not likely that it ever will be. With the develop- 
ment of demion theory in the last decade this conceptual problem has 
received more attention than ever before. Decision theory is both 
descriptive and norm-itivc, Ko attempt will be made here to survey 
descriptive decision theory.* 

•The most intensive survey of aornMtire decision theory i* that provided by 
Churchman (1961). A delailcd Ircntidenl is also provided by Edwards (1954), 
Luee and Raiffa (1957), Core and Zander (10S9), Chemoff and ^rosc8 (1959), 
ond Miller and Starr (1990), 
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The analysis of criteria of “best” choice on which we are about to 
embark is divided into three parts that are based on tlic distinctions 
described fay Luce and Raiffa (1957, p. 13) as follows: 

The Geld of decision making is commonly partitioned according to whether 
... it is effected under conditions of (o) certainty, (b) risk, or (c) uncer- 
tainty. . . .* 

(q) Cerfaintv if each action is known to lead invariably to a specific out- 
come ... 

(6) Riik it each action leads to one of a set of possible outcomes, each 
outcome occurring with a known probability. These probabilities are as- 
sumed to be known to the decision maker. ... 

(c) UnetTiainty if either action or both has as lU consequence a set of 
possible speciffc outcomes, but where the probabilities of these outcomes are 
completely unknown or are not even meaningful. 

In problems arising under risk conditions our analysis will reveal 
tliat an optimal decision can be defined as one which maximizes the 
mathematical expectation of valve or utility. The meaning of this 
criterion, however, depends on whether the objectives are defined 
guah'tativcly or guanftfarively. We will consider three types of prob- 
lem situations: those involving (1) only qualitative objectives, (2) 
only quastitaUve objectives, and (3) a combination of these. We 
will also consider the effect of other participants on the decision 
maker through counteractions and reactions to the decision maker's 
choice. 

In the discussion of uncertainty problem situations we shall deter- 
mine what adjustments in previously discussed procedures are re- 
quired. 

Before considering the various types of problems it is necessary to 
clarify the meaning of such expressions as qualitative and ^onfifafiVe 
objectivei, courses of action, and efficiency. 


QUALITATIVE AND QUANTITATIVE OBJECTIVES 

A qualitatively defined outcome is one which, following the choice 
of a course of action, is either obtained or not. There are no "in be- 
tweens.” That is, the outcome of an effort to obtain such an objective 
can be only one of two types; successful or unsuccessful. For ex- 
ample, if a person’s objective in going to a bookstore is to obtain a 

•Luce and RaUIs (1957, p. 13) indnde a fourth type which we will consider 
in Chapter 9: "a corDbinsUon of uncertainty and risk in the light of expenmen- 
lal evidence. This is the province of statistical inference. . . 
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certain book, then “obtaining that book” is a qualitative objective, 
since either he succeeds or not. Or, to take another ease, a person 
who needs $10 to pay a bill may go out "to obtain at least $10.” 
Although this outcome involves a quantity, it is qualitative in the 
sense used here because tlie person either succeeds in obtaining at least 
$10 or he doesn’t. 

A quantitatively defined outcome is one which is (or is not) ob- 
tained in various degrees. Tbat is, the extent to whicli such an out- 
come is obtained is potentially measurable. Fot example, if a person 
goes out “to borrow as much as posable,” the mount of success can 
be measured by the amount borrowed. Or, U & person tries “to reach 
a specified destination in Ute shortest possible time,” his success or 
lack of it is measurable by Ute amount of time required. It should 
be observed that a quantitatively defined outcome is really a set of 
objectives differentiated from each other by values along a specified 
scale. 

Qualitative objectives can, at least in principle, also be described in 
terms of a scale. For example, “to borrow at least $10” can bo speci- 
fied by the range of tlic scale of dollars from 10 up. Failure to obtain 
tills objective can be defined as obtaining less than $10. Dollars are 
treated qualitatively if there are two or more ranges on tie dollar 
scale within which the borrower places at equal value on all quanti- 
ties: if, for example, he places an equal value on “$10, $11, . . . $20 " 
tlien he treats “to obtain from $10 to $20” as a qualitative ebjecUvk 
On the other hand, if he places a different value on each point of the 
scale measuring the objective, he treats the objective quantitatively 
Thus the significance of the diatinctioo between qualitative and quan- 
titative outcomes lies in the measurement of efficiency of the courses 
of action relative to such outcomes. 


COURSES OF ACTION 

A course of action is not to be construed as a metbanisticallv « 
fled pattern of behavior. Varialiona in the action with res^Tt^T" 
certain physical characteristics may not change (he course of^ *• 
For example, “driving a car” may be desigaated as a course of T r 
There are many different ways of driving a car, but u / ^‘^“on, 

useful to group these into one class of behavior. DwnU- 
ations within the class, it can be distinguished j ^ 

■ as naking a bus.” A conne of aclion m., 1, ."“ s "l ul' ' 

.legree, of rigidity, depeodmg oo Iho 
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one purpose it may be desirable, for example, to distinguish between 
left-hand and right-hand driving. For another purpose it may be de- 
sirable to group uses of any Bclf-powered vehicle into one course of 
action. 

This flexibility of defining (or, more precisely, of individuation) 
also applies to physical objeeta. For one purpose an automobile may 
be considered as one physical object; for another it is a composite of 
many other units (e.g., wheels and transmission) ; and for still an- 
other it may be considered to be a part of a unit (e.g., a fleet of cars) . 

Notice the relativity of courses 0/ acfuwi and oufeomes. These are 
conceptual constructs which can be converted into each other, depend- 
ing on the interests of the researcher. For example, “sawing a tree 
may be considered as a course of action which yields the “falling of 
a tree" as an outcome. But “felling a tree" may be considered as a 
course of action which can yield the outcome “clearing a path." Such 
rclatirity of concepts is common in all areas investigated by science 
and hence does not present any unique methodological problem in this 
context. 

There are situations in which the decision maker does not directly 
select a course of action. Rather he selects a procedure or rule which 
perraiU the selection of a course of action in a specific context. This 
procedure or rule specifies bow the course of action to be taken should 
be derived from information available at the time action must be 
taken, though this information is not available at the time at which 
the decision rule is selected. Such a procedure or rule is called a 
rtrofegy and is itself a type of course of action. For this reason we 
nill use the terms course of action and choice to apply to both simple- 
choice and strategic-choice situations. 


MEAStJRES OF EFFICIENCY 

There are several tSTJcs of measure of efGciency, the definitions of 
which require the concepts of input and output. Input refers to the 
resources Vihich are consumed or expended in taking a course of ac- 
tion. A resource is anything of value; for example, men, money, 
material, time, or some combination of these. Thus, input refers to 
the “cost” of a course of action, where cost is used in a verj' general 
sense which is not restricted to monetary considerations. 

Output may be measured in tmms of either the resources which re- 
sult from taking the course of action or the psychological or socio- 
logical characteristics of the resulting state. For example we may 
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tiilk about t!ic money earned, the material produced, the time saved, 
or the amount of enjoyment which results from a course of action. 
Tims, output refers to the "return” or “paj’-off” resulting from a 
course of action. 

Outcomes — and hence objectiA’es (Le., desired outcomes) — ^may be 
defined in terms of either inputs or outputs, or both. For example, 
“to reduce production costs” is an input objective, whereas "to in- 
crease the amount produced” is an output objective. In the objective 
“to incre.ase profit,” the difference between input and output is critical. 

The tiT^e of measure of efficiencj' required depends on whether the 
amounts of input and/or output arc specified in the definition of the 
relevant outcome. The four possibilities are as foUows; 

(1) Speci^cd (or irrelevant) outputs and inputs. Here the measure 
of cITicicncy is simjdy the probability that Uie outcome will occur. 
For example, such an outcome would be "winning this game of chess.” 
This is a qualitative outcome, since it is either obtained or not. In- 
puts (e.g., time and effort) are not relevant Therefore the cfBcienciea 
of alternative plays or strategies arc measurabJe in terms of "prob- 
ability of success.” 

(2) Specified outpids and variable inputs. Here alternative courses 
of action are evaluated with respect to the inputs required to obtain 
a specified output, For example, if there are two different processes 
for producing an item, that process which requires least time or cost, 
or some combination of these, could be considered to be the most 
cflicient. Or the track man who runs a mile (specified output) in the 
least time (variable input) wins the race and is the most efficient. 

(3) Variable outputs and specified inputs. Here alternative courses 
of action with specified fixed inputs arc evaluated relative to the 
amount of output they yield. For example, the production process 
which yields the most product per unit of cost or time is the most 
efficient. Or the automobile which, starting from a stationary position, 
covers the greatest distance in a specified time can be considered to 
be the most efficient. 

(4) Variable inputs and outputs. Here neither input nor output is 
specified, and their difference or ratio Is used as a measure of efficiency. 
Tile course of action which yielda the greatest difference between out- 
put and input can be considered to be the most efficient. As already 
indicated, profit is measured by such a difference. Iletum on invest- 
ment, on the other hand, is a ratio of output to input. 

The discussion of efficiency could end here were it not for the fact 
that in most problem environments a course of action will yield dif- 
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FlcritE2.1. An efSciency functioti. 


fercnt inputs and/or outputs on different trials. For example, a pro- 
duction process does not maintain a constant output per unit of input, 
or a runner does not always complete a mile in the same time. In- 
puts and outputs vary, and this variation must be taken into account. 
In any particular case, if we determine the relative frequency with 
which the relevant inputs and/or outputs occur, we can formulate 
a probability-density function of these outcomes which we shall call 
an Sj^aency function. 

For example, a table which shows the percentage of light bulbs 
that can be expected to fail after various numbers of bours of usage^ 
a so-called "life table”— can be converted into an efficiency function. 
When plotted graphically, such a function might appear as shown in 
Figure 2.1. In this function, tlie area under the curve is set equal to 
1 so that the area between any two ordinates represents the probability 
of failure in the interval represented by the distance between the 
ordinates. As we shall see, efficiency functions ere required to define 
optimal solutions in risk problem situations involving quantitative 
outcomes. 


RISK PROBLEM SITUATIONS 
Qualitative Objectives Only 

The simplest type of problem situation is one involving only two 
possible outcomes, both qualitatively defined, and two courses of ac- 
tion. Let Oi and Oj represent the outcomes, Ci and Cj the courses 
of action, and Py the probaWlity that C« will produce Oj. An illus- 
trative situation of this type is shown in Table 2.1, which is called 
an cJTieicncv matrix. 

Which of the two courses of action shown is the better? On reflec- 
tion it becomes clear that this question cannot be answered unless 
we know which of the outcomes is deeired. That is, it is necessary 
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to know the relative values of these outcomes to the decisioD maker. 
If he desires Oi and not 0*, C| is the better action. On the other hand, 
if O 3 is hU objective, Ca is the better choice. 

As mucli ns scientists might like to avoid the necessity 0 / having 
to deal ivnth measures of even "relative” value, they have no alter- 
native in dealing with problem situations. It should be noted, how- 
ever, that although relative values must be assigned to the outcomes 
these are not the scientist’s atiIucs un1<»s the problem he is trying 
to solve is his oviTi. The values involved arc those of the decision 
maker. Since, to the scientist, an acceptable solution to the decision 
maker’s problem must be consistent with seicnea’s objectives, the 
scientist’s values are inYoh*cd indirectly. But we do not want to 
confuse the decision maker’s choice of a solution to his problem and 
the scientist's choice of a decision maker’s problem on which to con- 
duct research. 

Recognition of the necessity for determining the relative impor- 
tance of outcomes to decision makers has, in recent years, given rise 
to a ow ares at ectentiSc In the aext chapter 

we shall consider some of the problems and tcclminues wluch have been 
developed in this area. For the time being, however, wc will assume 
that we have a way of measuring the relative values of the outcomes. 
These wc shall represent by 

In the particular e-xample shown in Table 2.1, let Vi = 0.7 and 
r* a 0.3, Then wc can compute the sum of the "weighted” efficiencies 
of each course of action: 

i Ron 

/-I 


Table 2.1. An IixujmiATiTK Emciavcr Matrix • 


OulconiM 




Ot 

Oi 

Course* 

c. 

p„ = ao 

Pa = 0.1 

of 




Action 

Cf 

P« = 02 

Pn “ 0.S 


•If Ihc two outcomes fuc eshausllvc and exclusive, then llie sum of the prob- 
abilities in ciirli row must equal 10, otliMwiee not Id an^r t-uc, the sum of tbc 
probabilities in a column need not equal 14). 
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where Pi, is the probability that the lUi course ot action « ill yicM the 
;th outcome, and I’y is the relative value* of that outcome. For Ci: 

3 

^ p^JVi = 0.G(0.7) + O.-l(0.3) = 0.51. 

ForC2: 

2 = 05(0.7) + 0.8(05) - 05S. 

/-I 

The sum of the weighted cfTiclenclea of a course of action is the ex- 
pected (rflalifc) valite associated with that course of action. It 
seems reasonable to use “maximum cxpcclc<l value” as the crltcnon 
of “best” decision (Then Ci would be the one to clioo<e in the illus- 
trative problem.) But why should this criterion be useil? 

Justification for maximizing expected value 
The principal argument which has been used to defend the criterion 
of maximixatlon of expected value is ba«c<l on its apparent *'rca'*on' 
ableness." It seems obvious to many tint this outtM to be the cn- 
terion employed by rational bcin^. In feet, use of this criterion is 
often cited as a necessary (if not sufTicicnt) condition for rational 
choice. Despite the widespread acceptance by decision theorists of 
the superiority ot this criterion, science is ultimately obliged to find 
less intuitive grounds on which to support its selection of a criterion 
of choice, wliatever that criterion may be. Purely intuitive or rational 
grounds for selecting such a criterion f arc ultimately doomed in 
science. Such grounds must eventually have empirical as well as 
rational content. 

At present we can only suggest how these grounds might eventually 
be found. In order to develop such a suggestion let us first consider 
briefly the related question of how science might find grounds to sup- 
port the "ought” in an assertion such ns ".Y ouglit to do A in this 
environment." Churchman (1961) has made the most exhaustive 
analysis to date of how science might establish grounds for asserting 
this "ought.” In essence, his argument is aa follows. 

In establishing an assertion of the form “X is fc units" science must 
resort to a set of idealized conditions and operations called a stand- 
ard. (In Chapter 6 we will discuss such standards in detail.) The 

•We ahal! use the term i-olue to mean **Rlative value,” which is a measure of 
value assigned to an outconie that depctiils on the specific alternative outcomes 
that can occur in a particular contexL 

tFor example, see Chemoff (1954). 
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jissertion "X is k units,” iJjen, » equipoleni to the assertion, “A' wouid 
exhibit such-and-auch properties and behavior under standard condi- 
tions " Churchman (1961, pp. 17-lS) then reasons: 

Suppose wc could agree on the standard conditions for measuring the val- 
ues of outcomes. Then we would say that “.Y values an outcome to degree 
k" means “.Y would exhibit such-aiHl-such behavior in the standard environ- 
ment for value measurements." The present suggestion is to assert that the 
"ought” in a rccommencLition can be stated as follows: “Y ought to do A in 
this environment" means "Y would do A in the stsndanl environment that 
defines v-alue measurements." 

... If a scientist states that an executive should follow a certain course 
of action, he says in effect, "I have measured the values of the executive — 
or his organization — for the various outcomes that may result from his de- 
cisions These measurements predict what he would do if be were making 
his decision under the standard condition of value measurements. When. I 
say he ought to exhibit aueh-and-auch behavior. I mean that this is the be- 
havior he would e.xhibit if these standard conditions held. Of course he may 
not do what he ought to do; that is, the standard conditions may not hold 
in this enviromncBt.” • 

Just AS science must develop standard conditions for the measure- 
ment of value, so must ft eventualiy develop standard conditions for 
the determination of what criterion of choice a person employs. Once 
such conditions exist and we can use them experimentally, we can 
then translate “A dcctsioa maker, A', ought to use criterion C in this 
environment" into "A' would use criterion C in the standard conditions 
that define criterion preference.” 

The extensive experimentation directed toward determining how 
people actually do make decisions has, in general, failed either to 
define or to observe choice under explicitly specified standard condi- 
tions. As a consequence, the observations which have been made have 
many alternative interpretations, and little can be inferred about a 
general criterion of choice. 

At this stage we can only suggest n standard situation that might 
eventually be used to iletennine what criteria of choice a person ac- 
tually uses. Consider a situation in wliich there are m courses of action 
available (Cj, Cj, •• •, C,, C«), only ones of which can be selected 

at a lime. Let there be n pjwsible outcome.s (Oi, Os, •••, Oj, •••, 0,), 
which exhaust the possible consequence of a choice. Let Pu, the prob- 
ability that the tth course of aetkm will produce the jih aulcomc, he 

• In Chapter V ef fiw hoofe Cfiurchmaii attempts to rfeffne tAese tttndini con- 
ditioai. For An earlier effort to eatablieh such a Standard soc Churchman and 
.Arkoff (1917). 
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kiioT\-n pcrfcTily the dvrision maker. Further, the decision maker 
roust be completely aware of the relaUtc vatues, 1 1 , of the possible out- 
comes. He roust ba\e no prefereocc among Uie aUemativc courses 
of action for their own sake. Now there must be an explicitly fonnu- 
fated list of criteria of choice (CCj, CCs, — , CCm) Etich that each 
course of action is best by one and only one criterion, and for ever}' 
criterion there is only one best choice (i.e., the Cj and CCt are 
uniquely paired). Then the probability that the decision maker will 
select Ci would be a measure of his degree of preference for the cor- 
responding criterion of choice. 

Before such a standard could be used cfTcctiTcly, we would have 
to know much more limn we do now about controlUng euch variables as 
knowledge of probabilities, preferences among courses of action, and 
awareness of value.* 

If, in a parUcular problem situation, choices were always made so 
as to maximue expected value using this criterion, the average value 
attained would tend to this expectation and would be greater than if 
any other criterion were used. But what if we have a set of tleeision 
situations in each of which only one choice is to l>c made? Here too it 
can be shown that the average value attained will lend to be maximum 
if this criterion is used. 

Other criteria of choice wilt be considered in subsequent sections 
of this chapter. 

Quantitative Objectives Only 

"We have already considered how the cScieney of a course of action 
can be represented relative to a quantitati%-e outcome. The inputs 
and/or outputs associated wiUi a particular course of action will vary, 
but not all inputs and/or outputs will occur with the same frequency. 
TTien we should determine the probability associated with each pos- 
sible input and/or output and thus establish a probabilitj'-density 
function whidi we have called an ejETctency function. If the inputs 
and/or outputs are measured along discrete scales, the function will 
also be discrete. 

Consider firet two courses of action relative to one quantitative 
outcome. Then an eSriency fenetion would be obtained for each 
course of action: Jt{X) for Ci and J:{X) for C-. Now we need some 
criterion for selecting that coarse of action which has the “best'' 

•An «2ort lo constniet etandai^ for the* concepts can also be found in 
Qiurrfiatan and AtioS (IWT) and Cbortbsian (1D51). 



THE MLVNISQ OP "omMAL SOLVnOXS TO tEOBLEMs” 41 


riauEB 25. Two efficiency fvme- 
tions Kilh cqiisl means but un* 
cquat dispeniaos. 



efficiency function. Common sense seems to indicate that we should 
select the course of action which has the highest average efficiency. 
But on reflection we observe first that two courses of action may have 
the same average efficiency but have different dispersions or spreads. 
(See Figure 2.2.) In some situations the possibility of a very large 
outcome may be relevant to the choice made. For e-vample, s course 
of action which has some probability (though small) of resulting in 
bankruptcy for a company may be less desirable than one which 
has practically no possibility of such an outcome. 

More generally, wc can say that maximum “average efficiency” is 
an acceptable criterion of choice in problems involving one quan* 
titativc outcome only if the value of the outcome is a linear function 
of the measure of the outcome. Only if this condition holds is the 
course of action with the maiumum average efficiency nccessorifj/ the 
one which yields the maximum expected value. A linear valuo func- 
tion, however, is not very common. For example, outcome is fre- 
quently measured in dollars, and investigations have shown that the 
value of money is usually not a linear function of the amoimt [see 
Markowita (1952)]. 

Consider the two efficiency functions ehown in Figure 2.2. These 
two functions have the same average. To select tlie best course of 
action it is also neecssarj' to know the value function of the X-scale. 
In Figure 2.3 two possible value funcUons are shown. It should be 
noted that a value function need not be monotonicaily incnnising, as 
in Vjfi’). For example, the value of arsenic in a drug may increase 
A3 the amount of axsenic increases qp to a certain amount, but as it 
approaches a letlial dose the value decreases. 

if FjfA') holds, then Ct will yield tbe maximum expected value. 
If I'sf.Y) holds, then Cj will yield the maximum expected value. In 




tliose cases the expected value of a course of action, Ct, relative to 
a single quantitative outcome measured along an X-scalc is: 

/ V(X)/,(.V)iV, 

the sum of the weighted efficiencies. (In the next chapter wc will 
consider how the values V(A’) can be estimated.) 

Two or more quantitative ob{ectives 

If wo have a problem involvtog two courses of action and two 
quantitative outcomes, the efficiency matrix will appear as it did in 
Table 2.1, but the insertions in the cells will bo efficiency functions 
rather than single measures. Suppose, for example, that Oi in* 
volves costs with the associated objective “to minimiie costs,” and 
Oj involves time with the associated objective “to minimize delay to 
(say) customers.” In most industrial contexts these two objectives, 
it will be noted, cannot be pursued simultaneously. The consequences 
of this fact will become clear as we proceed. Now in each cell we 
will have an efficiency function which gives the probability of expend- 
ing various inputs expressed along a cost scale for Oi and a time 
scale for O 3 . We cannot "add” the efficiency functions for each 
course of action because the output scales are expressed in different 
units. ^ Since addition is necessary if we are to evaluate the courses 
of action^ relative to ait the relevant outcomes, we must find some 
way of either transforming one scale into the other or transforming 
both into some other scale. Such a transformation consists of equat- 
ing various quantities on one scale with quantities on the other, or 
converting both to an absolute scale of value. Since we do not yet 
have scales of absolute value, it is necessary first to transform the 
outcome scales (.Yj and X.) into each other and then to find the 
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Ficuu: 2.4. A tntufonnation function. 


JTifeg.. days) 



rvlative vnliiea of units along one of Ujcso scales. In transfoniung 
tlic scales wo must wciglit the tclativc importance of units of the dif- 
ferent scales. Successive units on one scale may* not be equally valu- 
able relative to units on the other. For example, a two-day delay 
to a customer may cost (in dollars) more Uian two one-day delays. 
Such transformation (or irnde-off) functions arc seldom linear. Fig- 
ure 2.4 shows such a function. 

It may be wonderwl why the two clficicocy scales cannot be trans- 
formed into value scales and then added, thereby eliminating the 
need of transforming one eSielCDcy scale into the other. The reason 
is that current methods can establish only tiio rclofive values of units 
on an cfScicney scale, not the alisolute values. Therefore, a tpecified 
relative value on one rfHciency scale is not necessarily equal to 
the same relative value on another cfneiescy scale. In order fo make 
the relative values comparable a transformation of at least some points 
on the efficiency scales U required. 

Once such a transfonuation function is obtaincil, it is possible to 
express both cQieicncy functions in terms of either scale and then to 
combine tbem by "addition” into an effecUvtnest function. For ex- 
ample, if /u(A‘i) and /is{A's) represent the efficiency functions of 
Cj for Oi and Oj, respectively, wlicrc Xt and A'» are values on the 
corresponding outcome scales, and if we obtnin a transformation func- 
tion [.Vj es (i(.Vj)], then the effectiveness function of Cj could be e.x- 
pressed as a function of A'l, AfA'i), where 

MA’O = r ""/iWrWA*! - 0)1 rfa.* (D 

•'a-O 

The effectiveness function could equally well be expressed in terms of 

A's. 

It & MsumeJ here t/iat oufeomes on Xj niKf are MepemteaL If this w 
not the cAflc, 

A(.Vi) /l(oV/ff(Xi -*)/olrfri. 
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Now, if a value function, FfJTi), is obtained, the expected value of 
Cl is BVfCi), where 

CT’(Ci) - f y(.X,MX,) dX,. 

It should be noted that this measure, expected value, incorporates 
within it a weighting of the dispersion (variance) of the efBciency and 
eCectiveness functions. That is, the expected values derived from (a) 
t^'o efficiency functions with the same average but different dispersions 
and (b) a nonlinear value (imcUon, will differ. The variance of the 
value of the outcome, therefore, has no significance. (The variance 
of the estimate of expected value, however, is significant.) Hence, 
one of the advantages of this measure is that it can stand alone; unlike 
expected efficiency and expected effectiveness, it need not be con- 
sidered in connection with a measure of variability. 

Mixed Outcomes: Qualitative and Quantitative 

Consider a situation involving two objectives, one qualitatively de- 
fined and one quantitatively defined. For example, in a business 
situation management may wish “to have the largest ehare of the 
market" and to "maximiae annual net profit.” The first objective 
involves two qualitative outcomes: 

Oi. to have a larger share of the market than any competitor. 

Oj: not to have a larger share of the market than any competitor. 

Tlie profit objecUve involves an outcome defined along a dollar scale 
^ necessary to find what amount of net profit is equiv- 
alent in value to attainment of Oj. This may be done by techniques 
to be considered in the next chapter. Let represent the amoimt 
of profit so determined. Then, for a course of action C,, the expected 
value is given by 

F-ViCi) = -t- - r(A'.)l + J r(X)f,(X) dX, (2) 

where /,{.Y) U the efficiency function of C, relative to the profit out- 
come, and V (A) is the value funeUon of the A’ -scale. 

In general, then, the qualitative objective is transformed to a value 
on the scale used to measure either efficiency (in the case involving 
one quantitative outcome) or effectiveness (in the case involving 
more than one such objective). The value of this transformed 
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quantity is obtained from the value function of the scale employed. 
Then for a specific course of action the expected values for the quali- 
tative and quantitative objecUves arc added. 

Counteractions and Iteactions 

In some problem situations a choice of a course of action (C<) by 
one decision maker will produce a responsive action {R») by another 
decision maker. Such is the case m competitive situations; /or e.v- 
amplc, in auction bidding, in pricing of products, in most athletic 
gtuncs, and in wars. The responsive action may increase or decrease 
the cfiieicncy of the first choice: cooperate or conflict with it. 

Siippcc that the first decision maker, A, has two choices, Ci and Cj ; 
and the second decision maker, It, has two responses, and J?*: two 
possible qualitative outcomes arc involved, Oi and 0?, which have 
relative values I'l and I'j to A. 

If B makes his clioice independently of A’n and a fixed probability 
cos be aasigoed to each of D's choices, then the expected value ot A'a 
choice of C« would be 

i’rcc,) - p(ff,)(^(Oilc,.ft,)i', + r(Oa|Ci,/j,)rs] 

+ P{Rt){PiOi\Ct.Ri)t\ -f miC., Ri)Vtl (.?) 

where P(0/1C<, Pt) is the probability that D, will occur if A selects 
C» and B selects P». If D'a choice is conditioned by A'a choice, we 
would substitute P(B||C|> and P(Pj|C«) for P(/?i) and P(R*) in this 
equation. 

If the outcomes Oi and Ot arc quantitatively defined along scales 
Xi and A's, the efficiency function for each course of action (CJ would 
have to be determined for each possible counteraction (P*). Let 
/i»(A'j) represent the efficiency function of C« for Oj if is selected. 
Then the eomposile efficiency funetion of Cj for Ot would be 

ftiXi) = PCfii IC|)/„(A-,) + P(P,lCi)As(A'i). (4) 

Using these composite functions, expected values can be determined 
in the manner discussed above. If the probabilities associated with 
the responses, P(R»), arc not known but the efficiency functions, 
/rt(A';l, arc known, wc have a problem which combines risk and un- 
certainty. Tliis type of problem will be considered below. 

If the probabilities, P(Pt), and the efficiency functions, /it(A'j), are 
unknown to decision maker A, he must make a decision under com- 
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plete uncertainty. In such a situatioii he has no basis for choice, and 
hence the choice must be arbitrarj'. If, on the otlier hand, A knows 
exactly what choice B will make, and what outcome will occur for 
any combination of his action and B% A will make his decision under 
certainty. 


DECISIONS UNDER CERTAINTY 


If a decision maker has a well-specified set of alternative courses 
of action for each of which there is one and only one outcome, then — 
jf he can determine the value of each outcome — he should simply 
select that course of action whose outcome has maximum value. Al- 
though in specific problems of this type it may be quite difficult to 
formulate the alternative courses of action, to identify the outcome 
associated with each, and to determine their relative values, the 
principle of choice involved n quite simple. 

How suppose that a second decision maker, B, is involved and that 
Ins interests are in conflict with those of the first decision maker, A. 
This is the type of decision situation which has been the subject of 
study in game theory. 


Intuitively, the problem of conflict of interest is, for each participant, a 
problem of individual dcci^n making under a mixture of nsk and uncer- 
Umty, the uncertainty arising from his ignorance as to what otliers will do 
CLucc and Raifla (1057, p. 14)3. 


In game theory' this type of problem situation is idealized in such a 
way as to transform it into a decision under certainty or risk. This 
IS not accomplished directly by assuming knowledge of what the op- 
ponent will do, but indirectly by assuming that the opponent has cer- 
tain information and that he is motivated in certain ways; that is; 
tliat he behaves “rationally.” 


point of fact, assumptions about 
sulTicient to eliminate completely the 
[Luce and Raiffa (1957, p. 14)^ 


the motivations of players are not 
uncertainty aspects of the problem 


Put another way, the central problem of game theory does not turn out 
to be what choice to make pven certain or probabilistic knowledge 
of the opponent’s choices, but rather what choices can the opponent 
he nssumed to make, given certain assumptions about bis state of 
knowledge and motivations. Let us be more specific. 

A game ’ is a decision aituation involving conflict of interest be- 
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twpcn two or more interacting decision makers in which the following 
conditions hold: 

(1) Each decision maker has available to him a set of two or more 
well-specified choices or sequences of choices called "plays.” 

(2) Every possible combination of plays available to the players 
leads to a well-defined end-slate (e g., win, lose, or draw) which termi- 
nates the game. 

(3) A specified pay-off for each player is associated with each 
end-state. 

These conditions can be represented in a pay-off raatrir in nhich tJje 
pay-off associated with each possible combination of choices is shown, 
as is done in Table 2.2. Inferences as to what the other player is 
going to do are derived from the conditions of the game, (l)-(3) 
above, and from the following two assumptions: 

(4) Each decision maker has perfect knowledge of the game and his 
opposition; that is, he knows the rules of the game in full detail and 
the pay-off functions of all other players. 

(5) All decision makers are ‘'rational”; that is, each player, gi>’en 
two alternatives, will select the one he prefers: the one which yields 
the greater value to him. 


Table 2.2. A Pat-oit Matrix 
(Relative Values of All Poeaible Outcomes) 
nayer B's Possible Plays 




fc* • 

- . 

6, 


Til 

r» • 

• IV • 

• Pu 

<lj 

Vji 

r« • 

• • 

• r,. 


Va 

Va • 

• Vi. • 

• r.. 


r«. 

r.t • 

• V./ • 

• v„. 


Player J 'a 
Possible Pla>'s 
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Id some games these two assumpUons do seem to rctiucc the situ- 
ation to one of decision under certainty. For example, conMilcr ft 
two-person game in which whate\er one loses the oliicr gains <i c., the 
sum of the gains ami losses from any pUy are ixpial to rcro). This i^ 
called a liro-person icro-rum come. Now 6iip|>o*c that the following 
pay-offs apply to A, the pay-olTs to P heing t!ic m'gati\o-< of tlio-c 
to A: 

Pb>er a 

Choices fci I'j 

Oi 1 4 

PUyer A 

d] 2 3 

Clearly, if A knows that B will select 6|, A should select n-; if B 
selects hj, A should select oi- Which choice will B make? Given 
assumptions (4) and (5) above, A may reason as follow*: 

B obsencs that it he selects 6| the most I can pin and be can lo»c 
U 2, but if he selects 6j I can gain and he eaa lose as much as 4. There- 
fore be will select hi and I should select c* 

In this reasoning B is assumed to act so as to mitiiroirc his maximum 
possible loss, and hence to minimixe A’a ma-ximum possible gain. A) 
on the other hand, acts so as to masimite his minimum possible gain. 
These principles of choice arc called tlic minfmor and nicmmin, re- 
spectively. 

Now let US look at B's reasoning about A: 

A observM that if he selects o, the least he can pin is 1, but it he se- 
lects <4 the least he can pin is 2. Therefore he will select a* and I should 
select bj. 

Note that the conclusions reached by A and D arc identical. Note 
also that neither player can improve his pay-oB by changing his 
choice if the other player does not change his choice. Ilencc these 
choices are said to maximite the “security level” of each player, and 
the combination of choices, a* and bi, is said to be an equihlinuta 
pair. Such a pair is called the solution to the game. 

Not all two-person xero-sum games have such a solution. This solu- 
tion can be shown to exist only if Uie pay-off matrix has an entry 
which is the highest in its eolumn zmd the lowest in its row .^n 
entiy of this type is called a taddle pobil. Several or no such entries 
may exist. For example, tonadcr the following case, in which a 
saddle point does not exist: 
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Pl.iycr 


Choices 

Ol 

at 


Vhycr D 

h b, 

2 4 

3 I 
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Again thp^e are pay-ofTa to A. 

Now suppose that A reasons about B ns be did in the prcvioiia case; 

B obscn'cs that if lie selects 5i the most he ean lasc U 3, but if ho 
selects bs the moct he can )i»e is 4. Tlicrefore bo will select bi and I 
should select o^. 

If B were to reason about A as he did before the result would be: 

A observes that if he edects Si the least he can gain is 2, but if he as 
Iccts Q] the least he ean gun u 1. Tlicrefore he will select Oi and I should 
select l»|. 

Note DOW that tlie ronelusioos rraebed by the two players are sot t}>e 
same. Furthermore, if the choice pair ot and 6t is made, B could 
improve his pay*off by ehangiog to bj, in which case A could improve 
his pay-off by changing to 0 |, in whtcli case C could improve his pay- 
off by cliAUging to b|, aod so oa without end. There is no cQullibriuni 
point, a point at which this reasoning can come to rest. 

This last ease ehon-s that some games cannot be reduced to deci- 
sions under certainty even with assumptions (4) and (S). However, 
game tlieory has shonTi how this last situation can be reduced to one 
of risk. The details of this reduction need not concern us here. 
Tlic important point for thb discussion is that the ability of game 
theory to convert a decision situation involving conflict of interests 
into a decision under certiunly, however limited that ability may be, 
depends on the assumption of complete knowledge on the part of all 
decision makers involved. It is sufficient for our purposes to observe 
that the assumption of perfect knowledge is seldom, if ever, justified 
in real problem situations. 

In most re.a! conflict decision situations the choices available to 
the players seldom completely determine the outcome. The choices 
and the environment cannot be defined so as to be sufficient for any 
specified outcome. Consequently, even if all tiio choices are known, 
the outcome is known only probabilistically at best. Furthermore, 
even ii the oxAcomea •were known vntb yAattrfi 

on them by the opposing players are seldom, if ever, known with 
certainty. For one or both of these rcMons the pay-olTs in real situ- 
ations arc almost always known with less than certainty. 
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It is for these reasons, more than any other, that eainc 
louatl so httle appheation. It has even been 

apply in contrived laboratory situaUona heeause ol the inah lity 01 
operimenters to control the values which suhjccU place on outcome . 
To say that the theory has little value in problem solving is not to 
say that It lias little valor. To Uie contrary, it is a very 
scicnliCc achievement lor reasons which have been well stated by 
Rnpoport (1959). , 

IVliat happens to the concept ol a Rnmc if the assumption ol pc - 
feet knowledge is dropped? Is it then a suHahlc model of decision 
under uncertainty? We turn to these questions in the next section. 


DECISIONS UNDER UNCERTAINTY 

If the assumption of perfect knoalcdgc in game theory is replaced 
with an assumption of complete ignorance, the result is a model of 
an uncertainty problem situation. We argued In the last section that 
to assume perfect knowledge of the players is to assume too much. In 
this section we will argue that to assume complele ignorance is to as- 
sume too little. The argument is based on the observations that In 
order to formulate a pay-off matrix for a problem situation some 
knowledge of the opposition (be it a decision maker or nature) >s 
necessary and that this knowledge can and should bo used in the 
decision process. 

In the uncertainty game it is assumed that neither player can as- 
sign probabilities to the possible plays of his opponent It is apparent 
that, if a player could assign such probabilities and could construct 
a pay-off matrix, he could try to maximize expected value; or if, 
the principle of insuHicient reason states, he is justified on the basis 
of his ignorance in assuming that his opponent is equally likely to 
select each possibility, he could still employ this criterion of best 
choice. OUierwise he must employ some other criterion. The three 
criteria which have been most seriously considered for this state of 
ignorance are the maximin utUity, niaimax regret, and llurui'cz a- 


Critcria of Choice under Uncertainty 

The most widely cited criterion is the marimm utility principle- 
Here the worst possible outcome (imnimum gam) that is associated 
with each possible course of action is identified. Then that course 
of action is considered best whose araociated minimum possible gain 
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is mnx-irnum. Jlorc precisely, let ot represent the strategies or courses 
of action available to the decision maker A, and let 6; represent the 
cJjoices available to an opposing deci^on maker or “nature," B. Then, 
according to the maxiinin criterion U>at decision is best for which 

max min (r(o„f<,)|, (5) 

where is tlie value of the outcome that follows the joint selection of 
oi and ij. 

The most common criticism of this criterion is that it is conseirative 
if not "pessimistic.” That is, Uic criterion is criticized for being ba«ed 
on an "expectation of the worst.” How can expectations be relevant 
in a situation in which prob.abiiilica are unknowable or meaningless? 
If this criticism is valid, then the possibility of prediction is implicitly 
assumed. 

Hurwicz (1931) attempted a gcncralitalioo of the maximin criterion 
to make it less pessimistic. IIis generalized criterion is as follows; 

ma.T [a max bj) + (I — «) min V{a„ 6^)1, (C) 

0, h, t, 

where a is some preselected number between 0 and 1. It is what might 
be called the index of opliminn.. If a is set equal to zero, (he Hurwics 
criterion reduces to the simpler form of the maximio criterion given 
above. If a is set equal to 1, the criterion becomes maximitatlon of 
maximum gain. The decision maker is supposed to select the value 
of A. 

This criterion has some undesirable characteristics even from the 
point of view of those who accept tiic meaningfulness of the assuinii- 
lion of complete ignorance in a game against nature.* But the.-fp 
need not concern us liere. It should be noted, lion ever, that a can be 
interpreted as an estimate of the probability that the opponent will 
select a course of action which maximizes Uic decision maker's loscps. 
Thus, the index of optimism can be taken to be a subjeclii-c jirob- 
ability estimate. 

A third choice criterion, minimax regret {nr wiiniMaj risk), was 
proposed by Savage (1951). 

Let the pay-offs to decision ranker A be tlie following: 

S, Si 

O) a 50 

ai 10 5 

•See, for example, nniinpr ami Manluk in Tiinlll, Coomlw, anil Davis (19M, 

pp. et-cs). 
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For each state of nature. S, and 5,. we can determine which course of 
action, Oi or o,. would yield the maximum gain and then Prepare a 
regret matrix in which we enter the difference between the P^y-oB 
mined and that which would have been obtained if the best a haa 
been selected. For example, the matrix given above can be trans- 
formed into the following regret matrix: 


Si S> 

ai 10 0 

at 0 45 

According to this criterion, A should select that o for which the 
mum regret is minimum, in this case Oi. More precisely, he shou 

min max R[af, Ij] = min (max max V(a,', 6,) — V(a;, (7) 

i, « »/ V 

where a! represents the choice that one would have made if he had 
known the choice bj in advance. Then, the regret function, R, is the 
difference between the value received from a given course of action 
and the value that would have been obtained had one knonm before- 
hand what choice opposition or nature would make. 


Selection of a Criterion of Choice 

The availability of the three criteria we have just considered presents 
the problem of detenniniog which one is most suitable in a specific 
situation. On this point Arrow (1938, p. 12) remarks: 

U is clear that we do not really have a universally valid criterion for ra- 
tional behavior [best choice] under uncertainty. Probably the best thing to 
be s^d is that different criteria arc valid under different circumstances. 

How can we determine which criterion is valid under any specific 
set of circumstances? TVhat is lacking at present is a meta-criterion 
which would enable us to determioe which of the available criteria 
of best choice is the best criterion. One can direct his efforts either 
to finding such a meta-criterion or to showing that the assumption of 
complete ignorance, which has led to the multiplicity of criteria, is— i° 
some sense— invalid. We shall take this latter course. 


The Assumption of Complete Ignorance 

Others have called into question the desi'robility of assuming com- 
plete ignorance of the probabilities of the opponent’s choices. For 
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example, Luce and Raiffa (1957, p. 299J, commenting on the situation 
in which the "opponent" is nature, tJjscix’c: 

A common criticism of such criteria as the maximia utility, minimax re- 
gret, nurwici a, and that based on the principle of insdhcient reason is that 
the/ are rationafired on some notion of eamplele ignorance. In practice, 
however, the decision maker usually has some partial information concern- 
ing the true state. No matter how vague it is, he may not wish to endorse 
any characteriiation of complete ignorance, and lo the heart U cut out of 
criteria based on this notion. 

The point to be m.Tdc, howev’er, is neitlicr that the assumption of 
complete ignorance is empirically fdlse nor that it is undesirable. It 
is a much stronger point that must be made: the assumption of com- 
plete ignorance is incontiaient witli the knowlcilgc required to repre- 
sent a problem in a pay-off matrix. 

I.*t us restate the misconception from which this inconsistency 
arises. In a real problem-solving situation the decision maker is not 
given the pay-ofi matrix; be must extract it out of the problem situ- 
atJoa itself. A prcformulsted problem, such as is used is a laboratoiy 
game, is a contrived exercise in which the decision maker is pres 
sented with the pay-off matrlt but not the information from which it 
was derived. The procedure by which he docs or should solve this 
exercise — it is not a problem— is not necessarily ttic same as what he 
docs or should do in a real problem situation that can bo represented 
by a pay-olT matrix. 

This can be seen by reference to a Iiypotlictical ndvertising problem 
dniwu up by Shubik (1955, p. 47): 

Two firms, A and D, each have a miUioo dolUrs to spend on advertising 
their products in a certain market area. TTiey can use die meda of radio, 
television, newspapers, magsrincs, an<l biSboanii For saplidly, »e will 
group these five alternatives into radio, television, and printed media. Tlie 
marketing research sections of e-ich firm work out the expected effect of any 
contingency. We will discuss the decision-making at firm A onty. A p-iy- 
off matrix of 4 X 4 is drawn up. This contains information on the 16 con- 
tingencies that imght arise if either firm spent all ita money advertising 
solely by means of radio, television, or printed metlw, or decided lo save the 
million dollars and not advertise at all. E-ich entry in the payoff matnx 
represents the amount of extra revenue above cost estimated under three 
circumstances (in millions of dcdlats). 

First we observe that there are many possible advertising media 
which are not included in this game. For example, the firms could 
advertise by use of skywriting, polnl-of-salo displays, and trai'cling 
exhibitions. In practice these could be excluded for only one reason: 
their choice by competition is extremely unlikely. To rxcludc them, 
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then, presupposes some knowledge of the probabilities of competitive 
choices. Now one might reply that this is doing injustice to an ex- 
ample and that the alternatives could be formulated so as to be exclu- 
sive and exhaustive. For example, one might formulate the alter- 
natives as: 

(a) Visual but not auditory. 

(fa) Auditory but not visual. 

(c) Visual and auditory. 

(d) Neither. 

Now it will be observed that there arc diRerent types of advertising 
that can occur in any of these categories. For example, "visual but 
not auditory" includes newspapers, magaiineB, billboards, skywriting, 
and 60 on. To associate a pay-off with such an alternative defined 
as "visual but not auditory," then, presupposes some knowledge of 
the probabilities of choice associated with these subalteroatives. No 
matter how fine the classification, this problem remains. Prediction of 
the outcome of any class of actions presupposes some knowledge of 
the likelihood of selection of the alternatives within the class. 

Even in Shubik's original classification many subaitematlves are 
possible in each category. For example, "radio" may mean spot an- 
nouncements or sponsored programs, day or night programs, many 
short commercials or a few long ones. To associate a pay-off witii 
“radio,” then, also nccesianlp assumes some knowledge of the dis- 
tribution of probabilities associated with tliese subaltersatives. 

These observations raise the question os to whether it is ever pos- 
sible to define a choice so that no variations of action are possible. 
Our earlier discussion of the nature of courses of action showed that 
a course of action is always a class of courses of action. Since no two 
different things can be identical in every respect, it follows that vari- 
alions within the class arc unavoidable. But one might still argue 
that, although there are diffcrcDCcs among the elements of the class, 
these differences may not affect the probabilities associated with 
possible outcomes, and hence these outcomes can be determined with- 
out any knowledge of the opponent. For example, it might be argued 
that the effects of all different kinds of “visual but not auditory" ad- 
vertising arc the same; that the outcome is independent of the "copy" 
(content of the ad) , its gcograpluc distribution, and who the advertiser 
is. Obviously wc can constrocl gaiaea (c.g., in which the 

outcome depends only on the play. That is, we can contrive situ- 
ations where an outcome is mechanistically or logically (i.e., determin- 
istically) linked to a course of action. The (jucstion before us, then, 
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can be reformulated as follows: can a problem situation as it arises 
in “reality” be adequately represented by a model in which choices 
arc deterministically linked to outcomes? Anyone familiar with the 
beha%’ioral sciences is aware of the fact that we are a long way from 
being able to define courses of actions and outcomes in a specified 
environment so that the connecUon between them is completely deter- 
ministic and (therefore) so that they are independent of such factors 
as who makes the selection. As long as outcomes, as we now define 
them, depend on variations among the elements in a class of courses 
of action or on outside factors, we cannot specify an outcome without 
liaving some knowledge of the effect of these variations. And this 
type of knowledge cannot be obtained without some knowledge of 
the likelihood associated will) various possible values of the action 
variables affecting the outcome. 

Nature as an opponent 

One might agree that wc cannot know an oppooeot’s choices and 
the pay-offs associated with joint choices unless we also know some- 
thing about the probabilities of the opponent's choices. One may still 
argue, however, that when nature Is the opponent such knowledge may 
not exist. To examine this ailment let us first recast the problem 
into a pay-off matrix which shows nature as the opponent. Let Cj, 
Cs, Ci Cm represent the choieea or plays available to the 
decision maker. Let Oi, Ot, * • 0* • • • 0, represent the possible out- 
comes of these choices. These outcomes ore produced not only by 
the decision maker's choice but also by the value of variables which he 
does not control. These can be called nature’s choices. Let the 
states icsuUing from nature’s choices be represented by Si, St, 

Then for each pair of choices (Ci, S») a certain outcome or etatc, 
0/, occurs which has an associated value to the decision maker, V/. 
If the decision maker can determine the probability associated with 
car}) of nature’s choices, i’lS*), then he can attempt to maximize ex- 
pected value. The question we want to consider first is: can the 
decision maker ever Imow what Uic S* are and not have some knowl- 
edge of these probabilities? 

In order to explain this kind of problem situation the following very 
simple analogy is sometimes used. Suppose tJjat nature consists of an 
um filled with black and white balls and its choices consist of random 
draws of a ball from the um. Suppose that tliis is all we know about 
nature’s choices; then, “obviously" we do not know the probabilities 
associated with these choices. But is this last supposition consistent? 

This model of a choice situation is net a model of a problem situ- 
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ation It fails to model reality for exactly the same reasons as the 
previously discussed competitive game does: it fails to provide the 
decision mater with the information required to formulate the prob- 
lem. Specifically, it does not tell him how it was determined that 
there are only black and white balls or even only balls in the bowl, 
and how it is known that the draws arc made at random. Consider- 
ation of how such assertions or assumptions could be justified in 
practice rev'eals that information would be required which could be 
used to formulate a reasonable assumption concerning the distribution 
of colors of the balls in the bowl. 

Kecently a defender of the bow] analogy suggested the following 
modification of the model. “The bowl is filled as follows; A biased 
coin is used. Each time it comes up heads a black ball is put in the 
bowl; otherwise a white one is inserted.” The situation is not changed 
by this modification. The information required to establish the fact 
that the coin is biased is sufficient to provide an estimate of the 
probabilities of white and black draws. 

In general terms, this argument has been to the effect that one can- 
not know what uncontrolled variables affect an outcome without hav- 
ing some knowledge of the likelihood of various values of these vari- 
ables occurring. 

A number of decision theorists have objected to the argument just 
given with the observ'ation that in many "real-life” situations certain 
information cannot be gotten directly but can only be inferred. The 
argument here, however, is not basM on the insistence of access to 
direct observation involving no inferences. To the contrary, in other 
places [e.g.. Churchman and Ackoff (1950)], 1 have maintained that 
all data are the result of inferential processes.* The argument is based 
on the observation that in such games as have been described the infor- 
mation used to match the characteristics of the game to the problem 
situation must be made available to the decision maker who must 
solve it before the game can become an adequate model of the problem 
situation. The argument thus far against the validity of the assump- 
tion of complete ignorance can be restated as follows: a problem 
situation cannot be formulated without some knowledge of the prob- 
abilities asfociated with possible outcomes. But up to this point wc 
have considered only situations in which possible outcomes are known. 
They may not be known initially, in which case research must be 
used to uncover them so that a fcirm»iU.tMjn ot the pcohlem 

can be made. Let us consider a situation of this type. 

•See also Chnre!mi»a (1961). 
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Objective and subjective probabilities 

Consider the following type of problem situation. In the develop- 
jjjcnt of a new niiJilniy ne.apon and policies for its use we luny not 
know who the enemy imII lie. or where, when, ntid under what circum- 
stances the weapon will be used. Here we seem to be in n state 
of cuniplctc ignorance. In tnililary projects of lliis type there are two 
fairly standard practices. First, a couafcrmcfisitres group is usually 
eslahlisiied n’li}e}> tric.s to develop instnimcnlj and tactics to combat 
the new weapon. The enemy is assumed to cstablUh such a group. 
On the basis of the countermeasure studies estimates are made ns to 
what tlie enemy is likely to do. Second, the weapon and its use are 
usually tricci in “map problems’* which arc simulated military engage- 
ments. T};c.se arc designed to in\*olvc the most likely enemies, the 
most likely locations, times, and so on. For example, Britain or 
Canada is seldom, if ever, used in tlic United States as the hypothetical 
enemy in sueli problems. Once a likely enemy is designated, exten- 
sive studies of its present and future capabilities arc used to estimate 
the likcliiiood Umt it will employ certAin spseiBcd ladies. 

It can be argued tliat the estimates of probability obtained is this 
way are not based on observed relative frequencies, and hence ore not 
objective probabilities. Tlits may be true, but so-called subjective 
probabilities siiould not be discarded lightly. Savage (1054) has de> 
I'cloped rt theory oi subjcctii'e probability whicli must be taken seri- 
ously. According to Luce and Raiffa (1957, p. 300), the school led by 
Savage 

. . . holds tlie view that by processing one's partial uiformation (as evi- 
denced by one's responses to a series of simple hypothetical questions of the 
yeS'OO variccy) oae can gcoerste an a priori prob.ibility distribution over the 
states of nature which is appropriate for nuking decisions. This reiiuces the 
decision problem from one of uncertainty to one of li'k. 

Churchman (lOGl, Chapter VI) puts both the traditional concepts of 
objective probability and the newer concepts of subjcelive probability 
to an exhaustive analysis. Jle eumaiaiitea (pp. 16^169) bis analysis 
as follows; 

We began with the hope th-st an objective measure of probabiUty could 
be found in the operations of countinK membership m classes. We conclude 
that no matter wh.it major aspect of this definition we choose to analyse, 
we are always driven to the necesaty of ioirodueing judgmeut; the opera- 
tion of verifying class niemh«vfr^ » based oa {ixdgsKnl; the operaiho of 
verifying the theory of sampling is based on Judgment; the verification of a 
theory of the generation of events is based on Judgment. 

Should we conclude that ultimately what probability really means is the 
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In this case the exact value of P need not be known; A need only 
know if it exceeds 0.25. 

It can be shown that in many, if not most, problem situations the 
probability associated with each posMble outcome need not be known 
exactly to select a course of action which maximizes expected value. 
In some eases only a ranking of probabilities is sufficient, and in other 
cases knowledge that the probabilities do or do not exceed some 
specifiable value is sufficient 
Errors in estimating probabtlKies 

We have argued that wherever one can define possible choices of 
nature or an opponent it is possible to estimate the probability asso- 
ciated with each choice. This does not mean that we can always 
obtain estimates in which we have confidence. The decision maker 
and the researcher may be very concerned about the possibility of 
serious consequences following from errors of estimation. There arc 
two ways of approaching this uncertainly of estimation; 

(1) If the decision maker must act — that is, he cannot delay the 
problem-then the estimates of probability should be made in such a 
way as to take into account the serious outcomes of which he is ap- 
prehensive. A procedure for so doing will be described in Chapter 8. 

(2) If the problem can be delayed, then either research should be 

conducted so as to obtain better estimates of these probabilities, or 
consideration should be given to the question as to whether the right 
problem is bemg studied. In such casra the right problem may very 
well be, 'T3ow can the problem situaUon be changed so that expected 
values can he maidmized?” Consider the following case. A company 
was trying to determine through research by how much it should in- 
crease its overloaded production facilities. The company produced 
equipment that is consumed in quantity only when general economic 
conditions arc good. It had developed an effective forecast of future 
sales assuming the continuation of prosperity. But since even a mU<l 
recession could have serious consequences to the business (particularly 
with expanded facilities), it was also necessary for the company to 
forecast general economic conditions. It did not feel that it could do 
this with sufficient accuracy, and it was pessimistic about the possi- 
bility that research would mprom this situation. This led the com- 
pany’s management to ask the qnestion, ‘TtTiat can we do to prevent 
a recession or depression from having such serious consequences on our 
business?” The answer management came up with was to find a new 
product which could be made with etisting facilities but which had 
a reaction to general economic eonditions opoosite tbeir 
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current product. Management directed research to be conducted to 
find such a product, and it was found. This reduced Uie seriousness 
of large errors in forecasting the probability of a receding market for 
the original product. Once this was accomplished, the question of 
expanding facilities was reopened and solved with confidence. 

It is quite common in the military, when it becomes clear that the 
outcome of a battle depends on what an enemy does and where what 
the enemy does cannot be accurately predicted, to attempt to develop 
equipment and tactics which are less sensitive to wl>at he does. TJie 
principle involved here is the same as that employed in the industrial 
example given above. 


BEST ANSWEKS TO QUESTIONS 

In order to apply the concept of “best decision" (as we have de- 
veloped it) to pure research, it is necessar>‘ to evaluate tlie losses 
(and gains) from falsely (or correctly) rcjectiog or accepting a “pure" 
research hypothesis or to evaluate the losses due to error in estimating 
the value of a parameter when this estimate may bo used for many 
purposes of which the researcher cannot be aware. Since these evalu- 
ations do not seem possible — or at least feasible— it appears that the 
pure researcher requires a criterion of “best answers to questions" 
which has no reference to outcomes of decisions and their values. 

Most pure researchers in the classical tradition feel that they have 
had such a criterion for some time: roinimization of error. They be- 
lieve, furtlier, that error can be defined and measured without con- 
sideration of consequences and Uicir values. 

IVe are going to argue to the contrary: (hat every concept of error 
contains an implicit set of aKumptions concerning the value of conse- 
quences. Prom this w e not conclude that the pure researcher must 
explicitly formulate consequences and their values — for this he clearly 
cannot do in many circumstances — but that he must measure and re- 
port errors in such o tcoy that they can be adjusted to suit citcutti- 
stances in which the values of consttfuences differ from those implicit 
in his measure of error. 

Since later chapters will consider in detail each type of question 
asked in pure science, here we will conridcr only one type of question, 
that involving the estimate of a parameter. Wc will consider only 
those aspects of estimation which are relevant to the argument. Ilsti- 
znstion will be considered In detail in Chapter 8. 
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Suppose that we are trying to estimate a parameter the true value 
of which is Y. Let yi represent estimates of this parameter. Tfie 
error of a particular estimate, y, clearly bccids to be the difTerence 
between T and y. Such a measure of error, }ion-evcr, has no opera- 
tional meaning because it would require a knowledge of the true value 
of the parameter. Consequently error must be measured in a way 
which does not presuppose knowledge of the true value of the param- 
eter being estimated. This is done by measuring properties of the set 
of estimates yielded by an estimating procedure, rather lb.an by meas- 
uring the properties of any one specific estimate. 

Such a procedure, however, involves two kinds of error, bias and 
vanabtittj/. The bias of an estimating procedure is defined as the 
difference between the expected value of estimates yielded by the pro- 
cedure and the true value. Bias in this sense can be calculated with- 
out knowledge of the true value of the parameter. There is a variety 
of ways to measure variability; the most commonly used is the van- 
onee, which is the mean squared deviations of the estimates from tlie 
true value. This measure can itself be estimated from ob8er%’ed data 
but cannot be known without error unless the true value is also 
known. For the purposes of this itiscussion we need consider only 
the first type of estimating error, bias. Note now that we cannot de- 
termine which of a set of estimates of a parameter’s value is “best.” 
We must settle for trying to define a best estimating procedure. Such 
a procedure may not yield the best possible estimate in any specific 
situation, but in some sense it >ne1ds estimates which arc best “in the 
long run.” 

The naive researcher might assert that the best estimating proce- 
dure is one that minijmzes bias and variability. There are problem 
situations, however, in which such an estimate is obviously not the 
best. For example, consider a situation in which for every unit of 
overestimation there is a loss of $1 and for every unit of underesti- 
mation there is a loss of $1^00,000. Conunon sense says u-e should 
deZibcrafely bias our estimating procedure toward overestimation so 
os to reduce the expected loss due to errors. Of course, it is just as 
likely that in another situation these penalties are reversed so that the 
bias should be in the other directimi. 

The pure researcher cannot produce an estimate which will mini- 
mize Josses due to estimating errora in all possible situations in which 
it might he used. He can, however, make and report his estimate in 
such a way that it can be appropriately adjusted (in a way to be 
considered in Chapter S). Therefore, from the point of view of future 
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application of the estimate in problem aituaUoQS it does not matter 
what the magnitude of the bias is, providing that it is made explicit. 

But what of using the estimate in answering another question (as in 
pure research) the consequences of which arc not known? The pure 
scientist will ordinarily use an unbiased estimate which can be shown 
to be best (in the sense of minimising expected losses due to error) 
only if the cost of error is independent of its sign (plus or minus). 
Wherever a pure scientist uses such an estimate, he implicitly assumes 
that this condition holds. The assumption is convenient in the sense 
that it simplifies estimating procedures. But it is important that con- 
venience not be confused with the quality of the estimate. 

What we have shown, then, is that an estimating procedure wbicli 
yields estimates that can be said to have no bias (and hence no error 
of this type) U not the best estimate in all situations in the sense that 
it win not always minimise the expected cost due to estimating errors. 

The costs due to cstimatiog errors vary from situation to situation: 
the function which relates cost of error to magnitude of error varies 
widely from one problem situation to another. Consequently, it is not 
possible to de\*cIop a criterion of best estimates in terms of error 
alone which will best serve our purposes in ever}’ situation. 

The discussion in Chapter 1 indicated that science seeks generality 
of its results, applicability over the widest possible range of condi- 
tions. This objective is not mol by any single estimated value of o 
parameter. Different estimates derived from the same data arc re- 
quired for different circumstances. Consequently, the objective of an 
estimating procedure should be to provide the information ncccssarj’ 
for preparing that esUmate in any specific situation which minimirc? 
the expected cost of errors due to esUciation. Ultimately, then, the 
best answer to a question is one which can be used in any problem 
situation to obtain n best solution. Truth and error of information 
ha^’c no meaning iDdependcntly of the way in which tho information 
is applied. "Correspondence with reality" cannot bo used to measure 
error, since reality is not known in a way which permits such compu- 
lation. Information conesponds to reaUty in any specific situation 
to the extent that it can- be used to accomplish our objectives in that 
situation; that is, to obtain best solutions to problems. 

It may seem heretical to assert that the quality of an ansn-cr to a 
pure rcscaTch question depends on how well that answer can be used 
h epp)}et} resc.ireh. This is to asserl that l.be uJllrastc r onfummaijon 
of science comes in the real world of problems and not in U»c ab- 
stract world of pure questions. 
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SUMMARY 


A problem was shown to consist of a decision maker, one or more 
objectives, two or more unequally efficient courses of action, a state 
of doubt, and an environment. Problems were divided into thr« 
classes: ceHainty, in which the outcome of any course of action is 
known; risk, in which the probabilities associated with possible mii- 
comes are known or can be esUmated; and unccrtatnfi/, in which these 
probabilities are unknown. In order to measure the cxpcctc v a ue 
of a course of action we found that it was necessary (1) to measure 
the efficiency of a course of action for each outcome in terms of pro • 
ability of success, and (for quantitaUve outcomes) inputs and/or out- 
puts, (2) to transform various ecalcs used to measure efficiency mw 
a single standard scale, and (3) to evaluate units along this scale. 
Combining these measures, it is possible to measure the expected va ue 
of a course of action. We took maximisation of this value to be the 
criterion of a best solution to a problem. 

Justification of this criterion is largely intuitive at present. 
did, however, outline a procedure for determining what criterion should 
be used. It consists essentially of determining what criterion woul 
be preferred under idealiied conditions in which the decision maker 
has perfect knowledge. It is to be expected that experimental work 
will move in this direction in the near future. 

We then considered a game as a model of a problem situation in- 
volving two or more decision makers with conflicting interests. Some 
such situations are transformed by game theory into decisions under 
certainty by assuming perfect knowledge and rationality of the de- 
cision makers involved. The assumption of perfect knowledge was 
criticised as being unrealistic. If it is dropped and we assume com- 
plete ignorance of opposition’s or nature’s choices, we have an un- 
certainty” game. AVe found this model to be inconsistent. It denies 
the decision maker access to information he would require to formulate 
his problem as such a game, specifically information about the prob- 
abilities of the possible outcomes. We argued that the formulation 
of a problem (e.g , identification of courses of action and possible 
outcomes) already uses estimates of probabilities of outcomes. These 
probabilities may be either objective (i.e., based on observed relative 
frequencies) or subjective (i.e., based on judgment). We considered 
the relationship between these types of probability and found both to 
be useful in science. 
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In effect, tljon, ne removed the aetxsstiy of defiling separately mcli 
uncertainty problem situations, because tlie uncertainty is contrived 
and not a characteristic of the problem Eituation itself. Analysis of 
various decision models showed that the probability estimates re- 
quired to maximize expected return may not need to be “point” esti- 
mates; rankings or ranges may be suflicicot. 

Finally, we considered the criterion of best answers to questions. 
We examined the assertion that minimization of error provides such a 
criterion. We found that for one type of estimating error, bias, mini- 
mization is not always desirable, since it may not minimize the ex- 
pected costs due to estimating errors. Error cannot be so defined that 
its minimization will always result in minimization of these costs. 
Consequently, the pure researcher cannot produce any one answer 
that is best as long as his criterion for so doing depends only on 
error. But since he cannot anticipate all the problem situations in 
whieli his answers may be used, he should give his answers in such a 
way that they can be adjusted to minimize the relevant costs in any 
particular situation. 
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FORMULATING 
THE PROBLEM 


INTnODUCTION 

In applied research H is necessary to translate the decision maker’s 
problem into a research problem. Tbb requires a complete identifiea* 
tion of the components of the decision maker’s problem, which are 

(1) the declstos maL'er(s)i 

(2) his (or their) relevant objectives, 

(3) the possible courses of action, 

(4) the context: those aspects of tie problem environment which, 
tliough not subject to the dedsion maker’s control, may aSect the out- 
come of his choice of acUon. Hiese may be 

(a) "acts of nature," or 

{&) acts of other deeidon maken: reactions or counteractions. 

Such identifications as are required may be very simple or quite 
complex, depending primarily on the problem context. If, for ex- 
ample, the decision maker controls a large organization within (or to) 
which others may react and which may be sensitive to general social 
or economic conditions, the mere formulation of the decision maker's 
problem may itseU require considerable research and consume a sig- 
nificHiit pw^cm ni iVie imai^ nvnvinV.e. 

Since our concern is not wjlh specific techniques of research, we 
shall not treat in detail the actual operations involved in problem 
formulation. But it will be nec«saiy at least to sketch these in order 
07 
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to have a background for discussion of the methodological problems 
involved in formulating the research problem. 

In this discussion of the aspects of problem formulation we will con- 
centrate on problem situations involving the operations of complex 
(i.e., organised man-machine) systems because the development ot a 
simplified representation of a complex situation is a part of 
formulation. Hence, to use a simple example would be to ignore tins 
essential feature of problem formulation. ^ .1 

Let us consider a decision maker whose choice will affect tue op ' 
ation of a complex man-machine system operating in a social 
for example, a government agency, a military arm, or an 
organization. The decision maker may be a government official a - 
templing to establish a public policy, a militarj’ leader seeking c 
best way to regulate purchasing of supplies, or a factory manager try- 
ing to determine hnw much of each of a set of products shou c 
produced in the next month. In each of these cases the environmen 
of the problem contains many forces which affect the kinds of choice 
which can be made and the kinds of outcome they will yield. It s 
generally necessary, therefore, to determine first how the environmen 
and the context are related to the problem under consideration. 
doing, the various components of the problem can be identified 
foundation can bo laid for subsequently establishing the relationship 
between these components. 


ANALYSIS OF TIIE CONTEXT OF THE PROBLEM 

Systems (and roost problem contexts) consist of a set of needs or 
desires and interrelated actiidlics connected by the flow of informa- 
tion which leads to decisions bringing about actions designed to yield 
some outcome that will satisfy the needs or desires. 

The "triggering” need or desire which it is the purpose of the system 
to fulfill must bo identified. In business, this may be a customer s 
desire for goods or acr\iccs. In government, it may be the publics 
requirement that mail be delivered, or that roads be available for 
travel. In the military, it may be the need to deter or defend against 
possible enemy aggression. 

How is information concerning the triggering need or desire con- 
veyed to the organiz.ation which can do something about it? Tlic 
forms and paths ot the communications should be identified Each 
message siiould 1>c traced until it Is disposed of or inactivated It 
may go Uirough many processes; infonnation may be abstracted, or it 
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may be combined with other information Ttie flow can be gho^m 
graphically by lines connecting '•nodes" at which the information is 
treated in some way. 

At each place where information is received a determination should 
be made ns to what is done with it, how long it takes to do this, and 
what the capacity of the node is for doing what it does, At some 
points very simple operations such as routing and compiling may be 
performed. At others information may be coileclcd, tabulated, anti 
analyzed. All the relevant informational inputs and outputs at each 
node should be identified. At some points along the information paths 
decisions are made. The location and nature of the decision should be 
identifietl. The instructional outputs should be traced in the same 
way as the informational inputs. 

The infonaation in tiie ayatem and the decisions b.ised on it even- 
tually make contact with the operations of the system; that is, they 
lead to action. The points of contact should be determined, and each 
step of the operations should be identified. The rate and capacity of 
operations at each stage should also be detensined. Tlic operations 
may involve such things as acquisition, transportation, conversion, in- 
spection, and packagbg of resources. 

The information on decision making and operations is usually best 
recorded and displayed in one or more flow charts generously sprin- 
kled with explanatory notes. 

In light of what has been teamed about decisions and operations 
in the system, determination should be made as to which messages 
have no effect on the acllviUes of intcresL These can be eliminated 
from further consideration. 

The activities which arc performed between control points (i.e., 
points at which decisions affect the operations) can be combined into 
one composite operation. 

The resultant inforraatSon-process-flow chart vdll show how the sys- 
tem operates, where and bow decisions are made, and what they af- 
fect and how. Such an analysis may frequently reveal problems of 
which the decision maker was unaware or even show that the problem 
which he thinks be has is quite differeot from the one he actually 
has. Such cases are discussed in Churchman, Ackoff, and Amoff (1957, 
Chapter 4). 

The complexity of an nnalj^is such ns has been described depends 
directly oa the complexity of the problem context. In the c.ase of an 
individual confronted with a commonplace problem— for example, 
which of two routes to take in driviiig to work— the same components 
are present: informational inputs, transmission, compilation, decision, 
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lion charts seldom provide reliable information in this regard; they 
may show who is "responsible” for control but not who exercises it. 


TIIE DECISION MAKER'S OBJECTIVES 

What is tbe decision makn- trying to attain by solving the problem 
at hand? And what is he trying to retainf Objectives may involve 
either getting something one does not have or giving up none or as 
little as possible of Bomething one does have. The decision maker’s 
desire to retain tilings he already has is usually formulated as 
restrictioni on possible solutions to the problem because they limit 
the set of courses of action which are acceptable to him. 

Despite a popular misconception to tbe contrary, objectives are sel- 
dom given to the researcher. The decision maker seldom formulates 
his objectives accurately. He is likely to state his objectives in tlje 
form of platitudes which have no operational significance. Conee- 
^ueolJy, ab/ectivea usyally have to be exlrseied by (he reaesKher. 
In so doing the researcher may well be performing his most helpful 
service to the dccisioa maker. 

Direct questioning of the decision maker seldom reveals all the rele- 
vant objectives. One effective technique for uncovering these oh- 
jectives eonsisU of confrontiog Uie dccblon maker with each of the 
possible solutions to a problem and asking him whether he would 
follow that course of action if the research results supported it. \Miere 
he says, “No," further probing will usually reveal objectives which 
are not served by the course of action. For example, in one case in 
which an industrial manager who wanted to diversify his company's 
production was asked if he would be willing to produce a product 
closely related to those be already made, he replied, "No." His ex- 
planation was, "That would be no fun; we know all there is to know 
about that product." In this way he revealed an iriterest in novelty 
and an opportunity to learn something new. It m worth cliecking 
such answers by asking subordinates and other associates what they 
think the decision maker would do in each situation. This may fre- 
quently reveal an objective not admitted by the decision maker. For 
example, in one instance in which a decision maker wpressed willing- 
ness to accept any of the alternatives proposed, each involving close 
association with a different person, his associates said that he would 
not accept one of these (or, if he did, he would sabotage it) because 
of his deep dislike for the person with whom be would be associated. 

This approach to uncovering objecUvra is based on the obser^•ation 
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that a decision maker and his associates are iikcly to be more certain 
about what they are willing (or arc not willing) to do than about why 
they are so inclined. Objcctive>« not only determine choices, but also 
are revealed by choices, though not necc!>*arily without ambiguity. 
A choice may be explained by many difTcrcnt objectives. Conse- 
quently, choices have to i»o studied concclivcly in order to find under- 
lying patterns. 

Objectives are hard enough lo define when they "stand still,” but 
many change frecpiently. Science has a long way to go before it devel- 
ops adequate techniques for delcnnining what objectives are relevant m 
any choice situation, and how and why they change. Nevertheless, it 
is necessary to formulate them as best we can. By failing to do so we 
do not avoid a research problem, we merely deal with it uncon- 
sciously. Such dealings do not lend themselves to sj'stematic im- 
provement through subsequent examination and analysis. Every- 
thing should be done to keep open and encourage this possibility for 
improvement. 


ALTERNATIVE COURSES OF ACTION 

Various alternatives may be revealed in discussing the problem with 
the decision makers and, in more complex situations, by analysis of 
the system or crgaoisalion involvctl. But aUemative courses of action 
may be os illusive as objectives. 

Essentially the task of identifying the possible courses of action 
consists of (a) identifying the variables that significantly affect the 
outcome of the problem, and (6) determining which of these can be 
controlled directly or indirectly by Uie decision maker. 

The variables which define the alternative courses of action may 
be defined either quantitatively or qualitatively. If the variables are 
defined quantitatively, each pcssible set of values along the relevant 
scales specifies a possible course of action. For example, the variables 
may be the amount of a resource purchased and the price paid. Then 
each combination of a possible amount and price constitutes a rele- 
vant course of action. Each possibiltly need not be explicitly formu- 
lated if the relevant scales and the feasible ranges on each are made 
explicit If the variable(a) are defined qualitatively, it is usuaUy 
necessary to specify each possible course of action explicitly. For ex- 
ample, the single relevant controllable variable may be the mode of 
transportation. The possible courses of action may then be use of 
an automobile, train, or plane. 
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choice should also be identified. 'Where these participanta arc govern* 
ments, military enemies, or competition, various types of '‘intclli* 
gence” may be retjuired. Available information of this type may be 
incomplete, and hence "reasonable assumptions’’ concerning such al- 
ternatives may have to be made by the researcher and decision maker 
working collaborativcly. In some instances it is desirable to set up 
a separate research or decision-making ^oup to pio!/ the role of the 
opposition in order to ensure against oversight of important counter- 
actions that might be selected. Military and management games * 
are used increasingly for this purpose. (We will discuss the use of 
such games for other purposes in Chapter 11.) 

"Hostile’’ or "cooperative’’ action may be taken by other than iden- 
tifiable participants; for example, nature may “select" bad weather, 
or the economic system may “decide" to recede. Such acts involve 
changes in relevant uncontrolled variables. Identification of some of 
these variables may require sophisticated causal analysis. Such anal- 
j-sis (also relevant to disclosing controllable variables) will be con- 
sidered in Chapter 10. 

Defining Objectives and Associated Measures of Efficiency 

The idestificatioa of objectives normally consists of nomtn^r the ob- 
jectives, but this leaves a great deal to be done. Tor example, in the 
military context one might come up with such objectives as mini- 
mising vulnerability and maximizing mobility, reliability, flexibility, 
or the state of readiness. In industral problems the objectives may 
involve maximizing productivity, profit, share of tie market, product 
leadership, growth, or security; or miounizing costs. In government 
the objective may be to maximize pubUc service, safety, or protection 
of property. It is a long way between naming these objectives and 
obtaining suitable measures for fbe degree to wbicb they are obtained, 
for all of them are quantitative in character. We shall discuss the 
problems of definition and measurement in Chapters 5 and 6, but some 
remarks are necessary here. 

The measurement of attainment of some of these objectives may 
appear to be deceptively rimple. Let us consider one concept which 
most people, particularly business executives, think they understand: 
profit. On the surface profit seems to be merely the difference be- 
tween income and costs. However, it is first necessary to determine 

•See ThomM and Deemer (1957) and Thomas (1961) for discussion of these 
Eames and their use in research. 
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whether one is coneemcd with gross or net profit: the ditTcrence be- 
fore B portion of the income is withdrawn for such things as taxes 
(i.c., gross profit), or only that income whose disposition remains in 
tiie control of the company (net profit)? Without considering the 
problems associated with tlie measurement of either type of profit in 
detail, let us enumerate some of the difficulties involved in finding a 
suitable measure for it. 

In order to measure the degree to wliich profits have been max- 
imized it is necessary to select a period with respect to which the 
measure is to be made. It is clear that in most cases one cannot 
consider only the short-range effects of a decision but must also con- 
sider the longer-rangc consequences. How far ahead should one look? 
Once this has been decided, it becomes apparent that a dollar earned 
today has a different value from one earned some time from now. 
This is usually taken care of by a discounting procedure, a procedure 
by which a future dollar's value is reduced by a specific percentage 
per unit of time. It is necessary, therefore, to select an appropriate 
discount rate. Furthermore, since many conditions can change wliich 
will affect what profits are actually made in the future, there is more 
uncertainty concerning future profits than there is concerning present 
ones. Therefore, it is necessary to take the uncertainly into account 
In evaluating possible future profits. 

Each of these questions — the planning period, the discount rate, and 
the adjustment for uncertainty— is very difficult to answer. The na- 
ture of this difficulty is discussed in detail by Churchman (1950). 

As indicated, in later chapters wc will consider how such objectives 
(ns well as other concepts) ought to bo defined and how measures for 
them ought to be specified. Nowberc in research, however, is it os 
difficult to approach doing as well as we know how as in obtaining 
adequate definitions and measures of objectives. The fact is that in 
much applied research the investigators do not explicitly define the 
objectives or develop adequate me.nsures of their attoinment, but 
rather they accept the use of convenient indicet. For example, con- 
sider the "slate of readiness" of a military force. Here researchers 
may use a measure of the average number of pieces of equipment 
which are availabte for use, and perhaps some measure of the disper- 
sion of the actual number available around this mean. Witli only 
minunum reflection the dcficieneiea ol such «n index become apparent. 
Obviously, readiness depends tra (amoag other things) the condition 
of the equipment involved. Equipment wliich is likely to be usable 
for only a short time docs not yield nearly as much readiness as 
equipment which can be expected to operate over an extended pcricxl 
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of time. Furthermore, readiness depends on the ability of the force 
involved to maintain the equipment in operating condition so that 
some measure of down time for repair should be a part of the measure 
of readiness. One could continue to point out shortcomings of the 
index, but it soon becomes apparent that a full-time research effort 
could be directed at only llie question of how to measure this state 
adequately. ^ . t . 

The research at hand would never be completed if we bad to solic 
fully ever}' research problem that arises on the way to solving the 
problem at hand. Therefore a critical problem in tlie strategj' of r^ 
search is where to cut off consideration of such problems as arise m 
defining and finding suitable measures for objectives. 

Wherever one stops and however one decides where to stop ia the 
pursuit of these subprohlems, it is clear that if future research is to 
do better than current research it is necessary to report explicitly on 
the considerations which went into the definitions and measures which 
are used. The deficiencies which are believed to exist should be made 
explicit, so that later researchers can begin where this research left 
off. In this way progress can at least be made gradually, and subse- 
quent researchers can be spared going through the difficulties encoun- 
tered previously, or at least can know specifically where to take issue 
with previous work. 

As a minimum the measure adopted in the research should be ac- 
ceptable to the decision maker. It must be one that he is willing to 
accept Hence it is important that the decUion maker actively par- 
ticipate in the development and/or acceptance of those measures of 
degrees of attainment of objectives which are to be used in the re- 
search. It should be noted that, despite the allusion to the contrary 
in the discussion of profit, decision makers are usually aware of the 
shortcoming of suggested measures of attainment of an objective. 

We assume in the remsuning discussion of this chapter that such 
measures have been obtained. But we will return to these problems 
of definition and measurement in Chapters 5 and 6. 


^tEASUBES OF PERFORilANCE 

In Chapter 2 the following types of problems and their require- 
ments relative to a measure of performance were considered: 

(1) One qualitative objective. All that is required is a jneosure oj 
efficiency for this objective. 
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(2) Two or more (qualitative objectives. A moasure of efficiency 
for each objective is required and a measure of the relative value cl 
each objective. 

(3) One guanlitative objective. A measure of efficiency involving 
an input and/or output scale is required and a measure of the rela- 
tive values of points on the scale employed. 

(4) T%eo or more q?mnhtaftt’e objectives. A measure of efficiency 
for each objective is rcquire<|, as well as a way of trans/onntnp units 
on the various efficiency scales employed into one "standard" scale, 
ami a measure of relative values of points on the "standard" scale. 

(6) Mixed objectives. Measures of efficiency and relative value are 
required and, if two or more quantitative objectives arc involved, 
transformations into a “Htandard" scale. 

There are two diSerent ways of handling the measure of perform- 
ance. In tlie first, tlie a priori method, the complete measure P is de- 
veloped before a model of the problem situation is constructed, so 
that the model ean take the form already referred to: 

where, ideally, P is a measure of evpected value. In the second, tlie 
a posterion, approach, the outcome is expressed either as a measure 
of expected efficiency for one objective or as an expected effectiveness. 
In the former case, the model is solved by optimising relative to one 
control variable for a variety of assumed values of the other control 
variables. This yields what is called an <Si<nent surface or curve 
from which a point is chosen by the decision maker. In the latter 
case, the transformation function is itself treated as a variable, so 
tJiat an "effectiveness surface" or "curve’' is obtained from which a 
point is chosen. In eitlicr case the choice of a point at which to oper- 
ate contains an impUeii transformation and evaluation of scales, as 
contrasted with the explicit translonaatiaa and evaluation in the a 
priori method. 

We will first consider fJje sirpe involved In the o priori method. 


A Priori Transformation Functions 

In order to demonstrate the varioua methods of tr-onsformlng one 
scale into another we shall consider silnalions which arise very com- 
monly in applied research, ones inrolving the conflict of two objec- 
tives. One of tliese objectives normally takes the form of minimising 
the expenditure of a valuable resource required to perform a desired 
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task. The other involves minimizing the inability to perform that 
task because of lack of resources. The first objective, then, involves 
the conser%'atioii of resources, and the second the avoidance of an un- 
desirable outcome. For example, in business a manufacturer may 
want both to minimize the cost of his operations and to fill any cus- 
tomer’s order when it arrives and, hence, minimize delay. The con- 
flict between such objectives arises out of the fact that the chances 
of avoiding the undesirable event can be increased only at tbe ex- 
pense of the resource to be conserved. The transfonnation problem, 
then, involves evaluating the "occurrence of the event to be avoided’ 
along the same scale on which the “resources to be conserved” are 
measured. 

Objective transformations 

The behavior of the process itself involved in the problem can be 
used in some cases to provide the information required to obtain the 
desired transformation function. 

Consider the following simplification of a real case. A merehandls- 
jrjg firm receives mail orders for goods selected by customers from a 
catalogue. The orders are filled by clerks who draw tbe reqrdred 
goods from stocks, package, and mail them. In this case the company 
involved wanted to determine how large a staff of order clerks it 
should maintain. Two objectives were involved: the company de- 
sired to minimize the cost of its operations (conservation ©bjecli^*e)f 
and it also sou^t to minimize tbe time tbe customer must wait for 
receipt of the goods ordered (avoidance objective). 

It seemed clear that to express the (negative) value of delay to tbe 
customer as a cost would be desirable. Consequently, the research 
team studied the system to determine bow much order-filling delay 
cost the company. First, records were wcamined to determ'me what 
proportion of orders were returned to the company by tbe customer 
because of delay and how this proportion varied for various delays in 
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FWvkb 3a. 



roJtauLATiNa the phoblem 79 


Fiouas 32. 



Average lime to delivery 


receipt of goods by tbc cu.«toioer TJiia asaiysis revealed a fuaction 
of the fonn shown in Figure 3 1 . That is, delays of up to a few days 
had practically no effect on the return rate; then for increasing de- 
lays the proportion of returns increased at a relatively constant rate. 
Beyond a certain delay the return rate remained relatively constant, 
rising almost imperceptibly. 

Kext an analysis was performed to determine whether the value of 
the goods was in any way related to the delay in sbipjaeat. No rela- 
tionship was found. This meant that the length of a delay in shipping 
did not depend on the sire or value of on order. 

It was now necessary to determine the cost of a returned order. 
This cost involves botfi direct outlays and customer dissatisfaction. 
Hence, an analysis was made of the 8.alvage value of returned orders. 
Salvage value was exprcssetl as a percentage of the sale value and 
was found to be relatively constant. 

By studying the distribution of the sizes of orders it w.'ts then pos- 
sible to determine the a\-erage dolhir loss resulting from a returned 
Order. To this e.vpected cost was added the cost of shipping itself. 
Finally, by mulliplynng this combined cost of return by the probabil- 
ity of return a function of the form sitown in Figure 32 was obtained. 
This, tlien, provided a transformation function in which the expected 
cost associated with various delays could bo obtained. 

Tlie question remained ns to whether or not customers were lost by 
long delays and, consequently, whether loss of future sales had to be 
taken into account. It was found, on examination of the records, how- 
ever, that delays of the frequency and magnitude actually experienced 
by customers did not affect the amount or frequency of their orders. 
Therefore the transformation could bo used without adjustment for 
loss of future business It was very likely, of course, that repetitive 
delays of long duration would affect future business, but the range of 
delays within which the company was willing to operate made the 
probability of a sequence of long delays very unlikely. 
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Tlip unilcriyinR principle in Uim procnJuTc is very eiinple: it at* 
tempts to determine vhal w tht «>/ rfsource to be comened 
in (?ie oemrrenre oj l^ic event In he oioi'/e</ nctcrinination of tlii’’' 
loss, hovrcvi-r. inn)' bo wry rotii|>]iratc«l or wen impractical. If, fof 
cNamplc. m tlic case just consuleml, cti‘tormrs ncru lost due to repe- 
tition of dila)s, it would liavc liecfi a niucb mure dilTicuH problem, 
Ihougli not necc«aari!y an impos«ible one. 

In anolbcr problem in which the event In lx: avoided was the can- 
cellation of a scheduled fli^it of a commercial airline it was not pos- 
fible (and, as it turned out, not nccessaiy) to determine the effect on 
future business of such a cancellation. A variation of Uie transfor- 
mation procedure just described waa u«c«l. 

Tlie problem of the commercial airline in%’oIvcd determining ho*'' 
many stewardesses should be located at each base. The objectives 
were “to minimize the cost of etewardwscs" (the size of the stewardess 
staff at each base) and “to minimize llie number of instances on which 
stewardesses were not available for a scheduled flight." If a steward- 
ess is cot available for a (light, it cannot be flown. 

An examination of the past hislor\' of the company revealed— to 
the surprise of the research team— that a flight cancellation due to 
lack of stewardesses had never taken place. This was surprising be- 
cause it couM he shown that with the existing staff the probability of 
running out of stewardesses was not negligible. It turned out, how- 
ever, tliat the stewardess admmi<trators at each base had advance 
knowledge as to when such a run-out would occur and therefore could 
take some preventive action. The administrator would obtain the 
aid of an ex-stewardess who lived io the city and who could be en- 
listed on a special emergency basis at an additional cost to the com- 
pany, In this esse, thai, the cost of a run-out was the cost of the 
emergency measure (expediting) which was taken. 

This cost is not a linear fonction of the number of simult-meous 
run-outs. That is, it becomes incrcasiogly difTicult for the adminis- 
trator to obtain emergency stewardesses as tJie number required in- 
creases. An examination of the records indicated that these costs in- 
creased approximately as the square of the number of stewardesses 
required; that is, quadratically. 

It should be observed that the cost of prevention might exceed the 
cost of the event to be avoided. But if it is the correct policy to pre- 
vent rather than incur the undesirable outcome, the “cost” of such an 
event as “running out” is the cost of preventing that outcome if the 
event occurs. 
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A similar problem arose in conn&tioo with production control in a 
pharmaceutical company. Here tlje objectives were "to minimize pro- 
duction and inventor}’ costs” and (again) "to minimize the lateness 
of delivery of customer onlcrs.’* An examination of past records in- 
dicated that delays beyond promised *!crivcry dates practically never 
occurred. The reason was that, when there was a shortage of a prod- 
uct relative to an order in band, a special production run was initi- 
ated and expedited under the direction of a manager eo as to be sure 
the promised delivery date was met. 

Here the cost of expediting involved two components: the cost of 
the labor (managerial and otlierwlsc) rcquifcd to perform the neccs- 
sar}' expediting, and the cost added to scheduled production by the 
interruptions in the sel>edulc. In this case the second component 
of cost was difficult but possible to determine. 

The general logic of this second procedure for making an objective 
transformation then is to dtlermine the ant (in terms of the resource 
to be conserved) of avoiding or jireventing the event to be avoided. 
This method assumes that avoidance is alwaj's possible. JustiHeation 
of this assumption may be dilBcuIt but is nevertheless necessary for 
the application of this procedure. 

Subjective transformations 

If an objective procedure cannot be used, then the researcher can 
ask the decision nakerfs) to sketch a iransfonnaiion function, using 
his best judgment. 

Tills subjective but explicit procedure at least opens the possibility 
for future objective evaluation of tiic transformation function ob- 
tained. It is important to remember that a decision involving quan- 
titative objectives measured on different scales requires transforma- 
tions. Transformations cannot be avoided, but they can be submerged 
in unconscious processes and hence can be kept from objecth'c esram- 
ination in the future. 

A Tnori Evaluation of Objectives 

Efforts to measure the relative value an individual places on ob- 
jects, events, or states «lale back at least a century and a half. In 
1789 Jeremy Benlham attempted to construct what he called a 
calcului oj values. Since then economists have speculated continu- 
oxaly on the measurement of utSity (their term for "value”). In 
the last twenty-five years psyebologists have directed their atten- 
tion to the measurement of ctlftiide*, a concept closely related to that 
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of value* But it has been only irfthe last decade that a major scien- 
tific effort has been made to develop and apply Wfh measu^. Bc- 
cent efforts have received their principal slimulalion from cither the 
work of von Neumann and Mo^nstem (1033) in the development of 
the theory of games, or the work of E. A. Singer, Jr., in the develop- 
ment of his philosophy of erpcrimcitfaliim (1021, 1930, and 1918).^ 

A completely genera! me.aswrc of relative value (one that is applic- 
able in all situations) has yet lo !>c developed. But progress has 
been rapid, largely because of a high degree of methodological self- 
consciousness which has characterized the work.t The measures 
which are available all make rather strong assumptions concerning 
the nature of values and human choices. An increasing number of 
different techniques of mc-asuring value based on different assumptions 
offer the researcher a growing store from which to make a selection. 

Contemporary scientists ioberit b strong disinclination to involve 
themselves in questions concerning values. The majority of leicntists 
and philosophers alike have ar6uc<l in the past that questions of value 
arc not reducible to or translatable into scientific questions. A minor- 
ity has argued to the contrary. But this argument is not immediately 
relevant to tlic problem at hand. The argument concerns the scien- 
tific determination of what people cu{fhl to value, not what they do 
value. Even If we grant that science ought not to concern itself with 
what people ought to desire^ we have set established the impropricly 
of science determining what people do value. Nevertheless, some of 
the scientific disbclination to deal with what people "ought lo value" 
mbs off on the question of what they do value. 

This disinclination is manifested in applied research by a tendency 
to consider only one — generally the most important — objective in seek- 
ing solutions to problems. This means that only a measure of efii* 
ciency need be used to find a “solution," no measures of value. The 
important point about such "simplification" is that it docs not obviate 
the evaluation problem. In effect such a simplification consists of 
an implicit assignment of maximum value to one objective and zero 
value to all other relevant objeclir-es. Such a procedure is likely to 
introduce more distortion of the real problem than even uncontrolled 
guesses by the decision maker as to the relative values of the ob- 
jectives. This simplification constitutes a resofufion, not a solution, 
of the evaluation problem. Evaluation cannot be avoided, but it can 
be pushed down below the level of research consciousness. 

•Bee Anw (1S58). 

tFor the mort complete niethodolooea} saalysia to date see Churchman (1061). 

tl am not willing to ^rant tiiis aasuinplion. See AeVoff (1019). 
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Since a large proportion ol the currently available measures of 
value are based on the work of tod Neumann and Morgenstem, we 
will first consider their measure aad the theorj' on which it is based. 

von Neumann-Morgenslem measure of utility 

In order to avoid introducing a netr set of symbols we shall trans- 
late the theory of von Neumann and Ikforgenstem into symbols W'e 
have already used, with only a few additions. 

(а) 0,, Oj, O 3 , ••• represent outcomes, objects, or anything to 
bo evaluated. 

( б ) Ol > O 2 means tlmt Oi is preferred to Os, and 0 % < Oj means 
0 - is preferred to Oi. 

(c) a, fi, y, • • • rejjresent real numbers between 0 and 1 . (These 
will be subsequently interpreted as probabilities.) 

(d) {a, OiiOa) represents a choice of eillier Oi with probability a 
or Os wdth probability (1 — o). ThU expression represents what is 
called a gamble. 

(e) Ol B O 3 means that neither is preferred to (he other; the deci- 
sion maker is indifferent as between Oi and Os- 

The following postulates make cx]dicit the assumed nature of 
evaluation: 

(1) For any Oi and Oj, cither Oi > 0., Os > Oi, or 0^ =» Oj. This 
asserts that any pair of outcomes can be ordered preferentially, that 
is, ranked. 

(2) If Ol > Oj and O 3 > Oj, then Oi > O 3 . Tliis asserts that the 
relation “is preferred to” is transitis'e. 

(3a) IfOi>Oj, thcnOi> («, Oi:Oa),foranyff. This asserts that 
it Ol is preferred to Oj, then certain attainment of Oi is preferable to 
any gamble in which there is some probability of less than 1 of getting 
Ol or getting a less desirable outcome with one minus the probability 
of obtaining Oi. Suppose, for example, that a person would ratlicr 
have a new wrist watch than a new fountain pen. Now suppose that 
lie is given a choice of a wrist watch or taking a gamble in which he 
obtains cither a wrist watch or fountwn pen. It is asserted that he 
would prefer the wrist watch to the gamble. (This assumption im- 
plies that the decision maker places no value on tlio act of gambling 
itself. This jmplic.ation will be discussed below.) 

(3b) If Ol < O 2 , then Oj < K 0,:0a). 

(4o) If Ol > Oj > O 3 , then there exists an a (0 < a < 1) such 
that (<*, Oi'.Os) > Os- Suppose that a wrist watch is preferred to a 
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pen and tlie pen is preferred to a pencil. Then this postulate asserts 
ttiat it is possible to find a gamble involving the watch and pencil 
that is preferred to the pen. 

• (4b) If Oj > O2 > O3, yien there exists a (0 < /? < 1) such that 

Ot > (/?, Oi'.Oz). Using the same sample as above, this asserts 
that it is also possible to find a gamble involving the watch and pencil 
such that the pen is preferred to it. (In general « will be “high” and 
^ will be “low.”) 

(5) (ff, OitOa) a (1 - n, 0j:0,), for any ft. This asserts that the 
order in which the choices are offered has no effect on preferences, 

(6) Oi:Oa):Oj} = («^, OjiO.), This asserts that probabili- 
ties can be multiplied, and hence compound gambles can be formed 
which obey the preceding postulates. Algebraic expansion of the 
left-hand expression gives the rigiit-hand expression. 

If the decision maker's values coafoms to these assumptions and if 
his choices satisfy two other conditions to be discussed below, his 
values can be measured as follows. 

Consider three objectives which are ranked by the decision maker 
in order of preference as Oi, Oj, and 0%. Arbitrarily assign a value 
to Of of I and a value to Oj of 0. Now find the a lot which 

oOj + (1 ~ 0)09 •• O9. 

Then Fj a «. Tor example, U the objectives arc (in order) watch, 
pen, and pencil; and if it is found that 

0.7V (watch) + 0.31’ (pencil) = l.OV (pen), 
then if V (watch) is set at 1.0 and V (pencil) Is set at 0, 

0.7(1.0) + 0.3(0) = I.OV (pen), and V (pen) = 0.7. 

In bis detailed evaluation of this theory Churchman (1956) has 
found "reasonable" grounds to question the general validity of each 
postulate cited above, and he has coosidered tests which might be used 
to establish or reject these postulates. Others have also called these 
postulates into question. Our concern here, however, is with the as- 
sumptions concerning the decision maker’s choices rather than ins 
scale of values. 

The von Neumann-Morgenstem procedure assumes that the sub- 
ject knows what the true probabilities are; tliat is, that the “chances” 
associated with the various outcomes which he believes to exist corre- 
spond exactly with the “objective" probabilities. Here again there 
is considerable experimental evidence to indicate that this is not true 
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in general.* Tins evidence lias driTCn some to develop value-meas- 
urement procedures whicli do not have this properly. One such ef- 
fort will be considered later. 

Certain practical consuiemtions in the use of sucli a measuring 
procedure are involved. The choices presented to the decision maker 
are difficult for him to comprehend. In addition, the procedure is 
awkward to extend to more tlian three objectives and becomes increas- 
ingly difficult as the number of objectives increases. 

Davidson-Siegcl-Suppcs measure of utility 

As one would expect, following the initial pioneering of von Neu- 
mann and Morgenstern into the no-man's land of values, a number of 
efforU were made to find measures which were in some sense better 
than the one presented above. One of Uio most sophisticated is the 
procedure developed by Davidson. Siegel, and Suppes. In a paper 
by Siegel (1056, pp. 212-213) the procedure is succinctly described 
as follows: 

»d Supp..,t in a study S.tb' to 

ol mousy m tks setuu of «n [J^„,'„^„..b.j/such uu evtnt i. d.f- 

s std srKrpiSSto. 

hold concerning famibar evenis. eg , heads . J ,jg qj, (tree 

m-ont umd bom 1, produmd by ?“rzoJ Toi™ »d .. S utbo, 
tom on tbe d,e lb. nomonm odtoto ZW » 

three faces ZEJ is engraved. Simiu a^lbbk-s were selected from 

xea, und QUO, OW on “K m,. SoL^ vduo. 

ab.o,t tvbo mpota thorn polm ■« b»'« the .upoototmu ot 

The dice note tooted “"h o«cb subj ^ .uJfflorenl uboot tthieh 

,eto uooomUon ™ urluell oo. uuuld b. the uinuor. The 

nonsense syllable be wouiu ocv i>u w Cniinwine section, 

use of these dice wiU be disc^«« ^ probabiUty-combinations pre- 

Careful and c^sidcred oonditioas. That h. when 

seated to a subject were asaurw y ^ assured of getting 

books were used as the b«>k he received was a funcUon of 

a book or books. The hjhiy involved in each choice, 

n/f of his choice behavior. entities, the subj’ect was given a 

When amounts of money gjart of tbe sesaon. He gambled 

sum ot money (usually one do possession at the end of the session, 

"'m iS^'dSrfhSto operTuonaUy how the subject's choices 

•See Edwards {19541. •Borne Experiments and Related The- 

. ’.ruumy .ud iubiootto Po*tbi«.y,- R.m no. <■ 

SUnTord Value Theory .\7VaIue of Nonsense Syllables ” Jvureel o/ 

tJ. A. Okie. "The Association 
Ci'fielic PiyekoloffV. S* 2“-2W. 
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determine ordered metric scaling is a one-person game * in wWch 

chooses between two alternatives, each of which is a probabUity-combination 

of two outcomes. The format for each offer is: 


AUernalive 1 A Uemalive S 
If event E oeeuTK yoagetw you gets 

If not-E occur*; you gfit t you get y 

The subject chooses the column; the outcome of event E detenmnes the row. 
Event E might be ZOJ, in which case nol-E would be ZEJ. 

Suppose to > * > y > 2. If the subject chooses Alternative 1, then 

(w,2;p) >(i,y;p). 

If u(to) is read as "the utility of to," and is interpreted as the subjective 
value of IP, ie., its worth to the person, then (1) can be written: 

p-a(tp) + (X — p)-«(0 > P-«(x) + (1 — ?)•«(!/)■ 

If p is understood to be subjective probability, and is known to be one-half, 
then (2) may be written; 


u(tr) -I- ti(i) > tt(*) + u(y), 

and 

«(«•) - u{z) > M(y) -nit), 

he,, 

ui > yt, when (io,2:p) > (r,y;p). 


(3) 

w 

(5) 


That is, IP and x dlfler in utility more than y and t. , 

It should be noted that the distances ate dirnUd distances. That is, aw is 
the negative of vii. To simplify comparisons, the convention has been adopted 
of always deriving the dishaoce from the more preferred to the less preferred 
entity. (For example, from (3) above we could get u(*) — u(y) > «(i) “ 

But since w > x > y > *, we multiply through by —1 to get (4) as shown.l 

From a practical point of view this procedure is reported to work 
well in laboratory situations. Siegel reports that, on the average, the 
time required to obtain the measurements for five entities is twenty 
minutes. There has, as yet, been no report of efforts to apply th® 
method in “real” situations. 

It should be noted that, though this procedure probably reduces the 
hazard of assuming the equivalence of subjective and objective prob- 
ability, it does not remove it. It also assumes, as indicated above, 
that choice is made so as to maximize expected value. 

Both the von Keumann and Daxridson procedures assume that no 
value is placed on the act of gambling itself. Evidence of such value 

• D. Davidson, 8 Siegel, and P. Suppes, *^0106 Experimenta and Related The- 
ory in the Measurement of Utility i»^ Subjective Probability,” Report bo 4. 
Stanford Value Theory Project, 1935 
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has been found by Hoyden, Suppes.and Wsish (1059), who then added 
tins consideration to their gambling model of value. They tried the 
“enriched rnodcl” on a group of sailors and college students. It fed to 
belter predictions for the former but woree for the latter. 

Churchman-Ackoff approximate measure of value 

The third proceflurc to he presented, unlike the preceding two, is 
not based on the concept of a gamble and hence makes no assumptions 
about subjective probability or marimiaation of expected value. Al- 
though developed independently, it resembles a procedure used In 
chemisliy for estimating values of a properly of each of a set of ob- 
jects where only comparative evaluaUons arc {wssible. The procedure 
first appeared in Ackoff (1933). Tlic underlying assumptions were 
presented in a subsequent paper liy Churchman and Ackoff (1954). 

The procedure is perhaps best understood by example. We will 
consider a case involving four outcomes. The steps are as follows: 

(1) Have the subject rank the four outcomes in order of importance 
and then assign numbers to each wbieh rcficct his relative cvaluaUon 
of them. Let 0|, Os, Oj, and O4 represent these DUteomes, ordered 
from the most to the least important 

(2) Determine which is preferred, 0, or the combination of O 3 , Os, 
and O4. If the combination Is preferred, then 

(2a) Determine which is preferred, Oj or the combination of 0$ 
and Oj. 

(3) Determine which is preferred, O 3 or tlie combination of 0# 
and O4. 

(4) Delcrmino whether or not the numbers aaBigncd in step (1) 
are consistent with the preferences expressed in steps (2) and (3). 
If they are, the procedure is completed; If not, confront tho subject 
with the inconsistency and have liini moflify either the niunbers or 
the preferences until they are consistent. 

Suppose, for example, that the subject lias assigned the numbers 
G, 4, 3, and 1 to Oi, Os, 0*, and 0«, respectively; and that he has ex- 
pressed the following preferences: 

Oi > (O 3 and Oj and O 1 ) 

Oi > (O3 and O4). 

The nurnbere arc inconsistent with both expressions of preference. 
If he changes the numbers to 8, 4, 2, and 1, the results would then 
be consistent. 
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This procedure has been adapted to situations involving large 
numbers of outcomes. Without such adaptation the procedure as de- 
scribed becomes unmanageable when more than about six outcomes 
are involved. The procedure for evaluating larger numbers of out- 
comes is as follows: 

(o) Select one of the outcomes at random. 

(fj) Assign the remaining outcomes at random to groups of approxi- 
mately. five each — the groups need not be equal — and add the out- 
come selected in step (o) to each group, 

(c) Evaluate each group by the procedure described in steps (1) 
to (4) with the following modification: after the subject has evaluated 
the first group, assign the number that he has associated with the 
outcome that appears in each group to the same outcome in the other 
groups. Subsequently, in assigning (and adjusting) numbers assigned 
to outcomes in the other groups be must leave this number alone. 

Reliability of this procedure can be checked easily by repeating 
the process, selecting a different “common" outcome and different 
groupings. Applications of these procedures and use with groups of 
decision makers are discussed in detail by Churchman et al. (1957, 
Chapter 6). 

The assumptions underlying this procedure are as follows: 

(1) For each outcome, in a set of outcomes in a specified situation, 
there corresponds a real nonnegative number V/, to be interpreted as 
a measure of the true relative value of 0/ in that situation. (Each Vj, 
then, is relative to the set {Oj] and the situation.) 

(2) If 0/ is more important than Ot, then Vf > and if 0/ and 
0* are equally important, then Vf = Ft 

(3) If V] and V» correspond to Of and 0*, respectively, then Vi plus 
Vk corresponds to the coinbined outcome Of and 0*. 

Tills additivity assumption will fail if Of and O* are logically in- 
capable of occurring simultaneously. For example, if Of is “to earn 
exactly $10” and 0» is “to cam exactly $20," these cannot simul- 
taneously occur and the assumed condition fails. 

In many cases where the outcomes appear to violate the additivity 
assumption they can be reformulated so that the assumption is applic- 
able. Consider, for example, a case in which an individual values both 
salt herring and chocolate cake taken separately, but not in cornhi- 
natioD. His desire for these foods may be formulated as “to have 
salt herring at a meal not involving chocolate cake” and "to liave 
chocolate cake at a meal not involving salt herring." The values of 
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these outcomes may satisfy the conditioa of Rdditivity, since they do 
not involve a conihinalion of (he foods at the same meal, hut rather 
ft combination of meals 

The additivity nssuniiitioD, not present in the von Neumann and 
Davidson procotlurcs, has the following corollaries: 

(за) If a, > Ok ftml Ok > Oi, then {0, and 0*) > Oj. 

(зб) (Oj and 0») = (0* and 0,). 

{3c) If (Oj and 0*) = Ot, then i', » 0. 

If an Oj exists sutisr 3 ’mg this last condition, then there is a zero 
point on the scale which is invariant with any transformation of 
tiie value scale. Tliia is not true for the two methods presented 
earlier. 

Another property of this procedure should be noted. It docs not 
assume that sub}cctive and objective probabilities correspond; prob- 
abilities are not in^*olvetl. It does, however, make the assumption of 
additivity and hcoce is applicable only to qualitatively dcHned out- 
comes. The gambling models are not ao restricted. 

In eflect, this method requires the decision maker to express a pref- 
erence between p-aired scU of outcomes. By the assumptions stated, 
these expresaions give rise to a 8>'8tcm of incqu.ilitica among the Vs. 
Thus the value of each outcome may not be uniquely determined. 
Consiiler, for example, a system of three outcomes, Oj, Os, nnil Oi, 
where the decision maker lias expressed the following preferences; 

( 1 ) 0 , > Os, 

(2) Oj > Og, and 

(3) Oj > Oj Og. 

This gives rise to the corresponding inequalities: 

( 1 ) V^■>Vs. 

(2) V, > V„ and 

(3) Vr>Vs+V,. 

Even if it is assumed that I, and 

(i) = 

there is do unique solution to this grstem. In fact, each possible 
triplet of values may be considered to be a point in three-dimensional 
space. It can be sbomi that tiie set of all such points is a convex 
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set. There are methods, lionwcr, tot cxpicssing analytically this 
solution Bct. 

Although the solution, using tins mclliwl, is not nccc>sarily unique, 
the method does provide two important inroads on the problem of 
measuring value. First, it provides a ‘■ysdtinatic method for deter- 
mining whether or not the decision maker’s espres-^ions of preference 
are internally consistent and consistent with his quantitative judg- 
ments of relative value; that »•», the numbers ns«igncd. Second, the 
procedure provides a method for detennining the bounds or limits on 
the estimates of value that arc consistent with the preference erpres- 
sions. In some cases, <!epcndmg upon the nature of the cfGciency 
matrix, eren though the solution set may i>c fairly large, any point 
in it will lead to the selcclion of tlie same course of action os will any 
other point. 

The techniques of linear programming can be used for finding all 
the possible sets of values which can be assigned to outcomes con- 
sistent with the pcefcrcDCcs expressed by Die decision maker. 

In the von Ueumann, Davidson, and Case * methods the judgments 
of the decision maker are applied to outcomes which arc assumeti 
(explicitly or implicitly) to be exclusive and exliaustivc; in the Church- 
man-Ackoff procedure the outcomes arc assumed to be neither. In 
many problems the outcomes as initially formulated may not satisfy 
the exclusiveness and exhaustiveness nquircmenl. Consider a com- 
monplace situation in which an individual wants black cofico an<l 
sugar. If obtaining coffee and obtaining sugar arc treated aa out- 
comes, they are not exclusive. To obtain a set of exclusive and ex- 
haustive outcomes we combine Uic two “primitive” outcomes into 
•’'compound’’' outcomes by means of a Boolean expansion : 

(1) Coffee and sugar. 

(2) Cofiec but not sugar. 

(3) Sugar but not cofiec. 

(4) Neither sugar nor coffee. 

In general, it there arc n primitive outcomes, they can be combineil 
into 2* compound outcomes that are exclusive and exhaustive. 

Suppose that the primitives are quantitative outcomes, say the 
amount of coffee and the amount of sugar obtained. Clearly, the 
value of an amount of coffee depends on the amount of sugar that is 
available; that is, these values are not independent, and hence they 

*Thi3 procedure is cooadered ia ttie next section. 
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cannot be added to obtain the value of a combination. Put another 
way, 

V(Ce.naS)9^V(C) + ViS), 

where V{C and S) is the value of an amount of coffee, C, and an 
amount of sugar, S; and l'(C) is the value of an amount of coffee, C, 
with no sugar; and V{S) is the value of an amount of sugar, S, with 
no coffee. Ilcnce, various combined amounts of coffee and sugar 
sliould l>c considered ns outcomes. This means that the outcome is 
multidimensional and that the value of combinations when plotted is 
a eurIncG or volume rather than a line. 

Case measure of rclaihe value 

Tiiis fourth type of measure sliarcn with von Neumann's theory 
applicability to hoUi fiuantitativc and (jualitativc outcomes, or to a 
combination of them. If transitivity is assumed, it also yields an in* 
tcn’al scale. If transitivity is not assumed, the measure can be used to 
determioe wheUicf or not the values oW-sinod are transitive. This 
procedure requires less complex judgmenU than tliosc required by 
either tiie von Neumann or the D.avid«on procedure. 

Consider a set of exclusive and cxhauslWe outcomes (Oi, Oa, 

0.)< Assume that one can obtain from a subject judgments which 
yield a set of probabilities (P|, Pj, •••, P.) such that the choices 
PjOj, PjOj, •••, P.O* are equally prcfcfre<l.* Tlien, on the assump- 
tion that the subject is tryiog to roaximiic expected value, we can 
conclude that 

Pil', -/'jP, •=•••'• PhI'.. (G) 

where Vj is the relative value of Oi, V- b the relative value of O 3 , 
and eo on. 

Now let » K, where K fa some arhilmrily selected constant, 
say 3.0. It follows from PiVi “ that 


Similarly 



(7) 


( 8 ) 


• Id each case where P, refers lo the prebabrnty of altainiag Oj. tlie allcmali>-o 
outeome wiUi probability (1-P,) is e»iimed_ to be in effect: "no change in the 
present state of the enrironroett” or “mamtaining the status quo." 
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or, in general, ^ 

V,. 

(9) 

Then 

Pi Pi 

r. + -v, + -r.+ 


( 10 ) 

From this we obtain 



/ Pi Pt 


( 11 ) 


Pi Pl\ 

( 12 ) 


Once the numerical value of Vi la thus cstablishotl, the relative 
values of Vj, V}, • • • I V, arc easily obtained. 

For esample, consider four outcomes in which the preference equal- 
ities are 


Then 

Hence, 

Therefore, 


and 




O. lOi - 0.30, - 0.20s - O.IO 4 . 
Vt = iV, + 4V, + fV, - 1.0. 
Vi-l/(l + J + 2 + .l)--gV. 

V, “ (J)(!^) ■= ^ 

V, = 2(*) = ^ 

' < = it 


it 


1.0. 


If (hree outcomes are mv<rfvcd, we could proceed by finding the values 
of Pi, P,, and P 3 Euch that 

PiO, = P 2 O 2 and P,0, = P 3 O 3 (13) 

or 

PiOi = P^i and PaO, = Ps'Oa. (14) 

For the first case, as n’e have shown, 



(15) 



In tlie second ca.se, 
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(10) 

and 

. p, 




(17) 

Hence, 

,, Pi P, „ Pi 




(18) 


Now, if we proceed by using Loth sets of comparisons and if the 
rankings of tiie two sets — Pi, Pj, P* and Pj, Pj, P,' — are identical 
with respect to the subscripts 1, 2, and 3, tlie preferences are tran- 
sitive; if not, the preferences are rot transitive. For e.<camp]e, Pi > 
Ps > Pi and Pi > P* > Pi yield transitivity, but Pi > Ps > Ps and 
Pi > Pa' > Pj do not. 

In addition, by comparing tbe values Ps and Ps wa obtain a good 
measure of the reliability of the judgments of the subject. 

The fundamental question which this and tbe other methods of 
measuring value raises concerns the possibility of developing a pro- 
cedure which makes no assumptions concerning the nature of values 
or tlie criterion of selection actually used by the decision maker. 
Churchman and AckofT attempted to define such a procedure (1947). 
Churchman (1901) has re-examined this suggestion in light of the 
recent developments of such measures os have been discussed iicre. 
The procedure requires the ability to measure such properties of the 
evaluator as his fanuliarity with and knowledge of the courses of ac- 
tion available to him. At the present Ume these are at least as diflicult 
to measure as is value. The procedure, then, cannot be applied in 
practice at present, but it represents a firat effort to provide a com- 
pletely general standard for the measurement of relative value. 

Minimal information required on relative values 

Although the criterion of maainaum expected value is defined as 

max 2 where 2 (1®) 

it is not always necessary to know the numerical value of tlie V/s in 
order to find the course of action which yields this maximum. If only 
two outcomes are involved, 0* and O*, and they are defined so as to be 
exclusive and exhaustive in the relevant environment, then we need 
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only know if Vi > Vs, Vs > V„ or Vi = V,. This can be shown as 
follows: 

(1) Since Oi and Oi are excluave and exhaustive, En + Eis = Esi 
+ Eis s= 1.0. (By convention let Ea>Eii‘, then, Ejj > Ej*.) 

(2) Therefore, En — Bji = Ea — Eis- 

(3) Now, EiCt) >E(Cs) if E„y, + E,,V,>E„V, + Es 5 ri, or 

E„y, - E=,r, > B,.yj - B„y, = yi(E„ - Sn) > vdE,, - b„). 

(4) But, since Eu - Esi = Bsj — Ets, tliis last condition reduces 
to V, > Vs. 

Hence, if V, > Vj, E(Ci) > E(Cs). 

If Bn = E 2 i, then E 22 = E ,5 and E(Gi) =£(0:) whatever tlie 
values of the V/e. 

If as few as three objectives are involved, however, we can siiow 
that is may be necessary (and it is always sufficient) to be able to ex- 
press the ratio of differences in the V/s ns numbers, and hence as 
values on (what will be shown in Chapter C to be) an inten’al scale. 
Consider the following case: 



1 

Ot 

Ot 

ZEi, 

Cl 


En 

En 

1.0 

c ,' 

1 E ,» 

En 

En 

1.0 


wliere Eu > Ej,, Ess > E,,, and Ei, > Ess’, and Vj > Vj > Vj. We 
wish to determine whether 

(1) E(C,) - E(C,) = y,(B„ - Ej,) - I'afEss - E,j) 

+ r,(Eu-E2j)>0. 

(2) Consequently, Vi(Ei, — E«) + V*(Bij — Ea) 

>n(E2,-E„). 

(3) But (En - E-O + (£« - E--) + (£„ - E**) = 0, and there- 
fore (E„-E„H. (E„-Ea) ^-(Eis-Ess) = (E-. - £«). 

(4) Then, substituting in (2), wc get VifEn - E-i) + VstEia - 
E-a) > I'sfEu — Eji) -j- Vj(Eij — B-j), and lienee 

TifEn - E„) - rj(E„ - E«) > r,{E„ - Eos) - lafE.s - E^.) 
(Eu - Eji)(r, - I'j) > (Cn - E,j)(rj - Vs). 

(5) Finally, 

(Vt - y»? g» - Ess 
(V,-V,) B„-Es.’ 
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Since the £y'e arc numbers, the right-hand tenn will be a number, 
and it is necessary to know whether the ratio of tiie differences of the 
r/a is greater than this number. Now it remains to show that, if the 
ratio of the differences of the Vf can be asserted to be larger or 
smaller than some specified number, this is sufficient for selecting that 
course of action which maximises Ujc expected value, given the effi- 
ciencies, Et].* 

Ixt |0/) l»c a finite set of ij exhaustive and cxchi«ivc objectives or 
cotisequenecs. Tlien, 2 1-0. By convention, let 

Oi > Oj > • • • > 0„ and lienco Tj > V'a > • • • > 1'*. 

Now compare two courses of action, C\ and Cs, to deterraino 
whether the expected value of Ci, £(Ci), is greater than or equal to 
the expected value of Ca, E{Cx). If E(Ct) - E{Ca) < 0, then Ct 
should be selected. If C((7i) — E{Cs) >0, Ct should be selected. 
If E(Ct) = SfCs), it makes no difference which is chosen. By con- 
vention then, if E(Ci)^EiCt), select Ct. 

Now, 

B(c,) - W,) - t / - £ B,ir, 

- r iEu - E„)V, + - Ex„)Vn 

= E (Eu - Exj)Vf - r. E (Eu “ £*;•) 

j-i /-I 

^''E(E,i-E7tm-i\). (20) 


Then, if Vi = Vt, from ccjualion (20) and the fact that ri^T’s> 
£(C,) - £(Ca) =• 0 (21) 


and either Ci or C- (say Cj) may be dropped from consideration. 
If Vi > V„, from equation (20), we obtain 


E(C.) - E(C,) 
Vl - Vn 


Vf - Vn 


•The /allowing proof j« due to ay coUeegra, Pelrr Rahbum (1961). 


( 22 ) 
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If all tlie r/s arc measured on an interval scale, then all terms 


V,~Vn 

Vi - v» 

are invariant on this scale, and their values are knonm. Then 


( 23 ) 


( 21 ) 

i=l 1 1 - 1 » 

is known, and moreover is invariant. 

Since (Ti — V«) > 0, it follows from 

that , 

B{C,) - B{C2) > 0. (26) 

It follows from 

X;(E.,-E„)^^<0 (27) 

J-l Vt — 

that 

£(C,) - E{Ct) < 0. (28) 

This proves the sufficiency of the ability to express the ratio of dif- 
ferences between V/s as a number for selecting that one of two 
courses of action which maximises expected value. If more than two 
courses of action are involved, comparisons between them can be 
made pairwise, and the surviving one is "best.” Hence this ability is 
sufficient for any finite number of courses of action. 

In some cases involving three or more outcomes it may not be neces- 
sary to have more than a ranking of the V/a ia order to select the best 
course of action; for example, if for Uirec outcomes, En > Est, Ets > 
Ej;, and E 59 > Ejj. Here, 


E(C,) - E(C2) = r,(E„ - E„) 

+ V'i(B,! V,(E„ - E„) > 0. (20) 

Since 

(E »3 — Ejs) = (Sti — Eji) -f- (Ej 2 — • £22)1 (^®) 

it follows that 

ITiff'ii ” ^^ 2 i) + Ta(f?ij — Exti > — Eu)\ = F».CEti — £21) 

+ V3{E„ - E*2) > VafEu - jE«) + ^(£^12 - Eaa). (31) 
Then, clearly this last inequation bolds if Fi > V 2 > V*. 
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In some cases where a ranking of tlie T's is not euiEcient to deter- 
mine whether B(C|) - f?(Ca)^ 0 , a ranking of adjacent differences 
of the Vf, such as Ki — Fj > I'l — rjjWiil be eufScient.* Tor example, 
suppose that the previous illustration is tnodificcl so that En > Ezu 
^S 2 > £i:, £^i> > Es 3 and £« — £21 > t'u - Ets. Furtiier, suppose 
that in addition to I'l > Fj > F* we know F| - F 3 > Fj - Fa Here 
we write 

K{Ci) - J:(Cs) - (E„ - E2,)(Vi - F,) - (£-,j - /;j3)(l'j - I’a). (32) 
obtained from 

E{Ci) - E{C,) = (En - £«)F, + - AjIFj 

+ (K.s-iM)Fs (33) 

by setting 

~ “ — (E|i — £ji) — {^ij — Ets). (34) 

Hero E(Ct) — E(Ct) > 0 Is the same as 

{Eu - ^ai)(F, - F,) > iCii - EmXFj - 1',), (35) 

which (i ^•aIid since 

- Eti > E,9 

and 

I', - V, > F, - 

But wjUioul Ujc in/onnalion that F| - 
not conclude that 

(£'j, - r 3 ,)(F, - V 2 ) > (K ,3 - HaalCFj - Fa). (3S) 
The point of these obsen'stions is that, once we know what the 
values of tiie Eu arc, we can detennine the minimum amount and kin<l 
of information required about the F/s. Then, by waiting until we 
know what the Ey’s are, we may reduce tlie work required in evaluat- 
ing objectives. Correspondingly, if it la easier to measure the V’/s 
than the Ey's, we siiould wait until the F/« been determined to 
learn what information about the Bf/a is required. 

A Foiteriori Approocli to Decblon Making 

fn (fie 0 po$(enon' approach <0 decisibn ma&i'ng ihe ouCcoraes are 
expressed only in terms of the effidency or effectiveness of courses 
* Tlus is railed an ordered nteirie teak and is also discujsrd in Chapter Q. 


- > 0 (30) 

- r, > 0. (37) 

- Fj > F, — F*, we can clearly 
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of aclion, and the decwion maker msist felect that course of action 
which in his judgment yields the best combination of these measures, 
where "best" « not explicitly defined. 

The first procedure (use of efficiency cur\’cs) which we will con- 
sider employs only measures of efficiency which arc expressed on dif- 
ferent scales. The second procedure (retrospective optimization) re- 
sembles the first except that past decisions are used to determine 
wlial trade-ofis (transformations) would have made these decisions 
marimize expected eficcliveness. The third procedure (use of cffec- 
liveness curves) deals with the ratios of measures of efSeieDcy, whicli 
measures are not made explicit but arc assumed to be expressed on the 
same scale. 

£fficlen <7 curves 

This approach has been deseribc4l by llanssmann (1961, p. 237) as 
follows: 

Suppose there arc only two eonflicting objectives. Then the levels of 
achievement, 0i and Oj, arc dependent on each other. This depesdesee may 
be exUbltcd by a eun'e (lec Rgute (3.3]). It is erpied thst-nit least sub- 
consciously— every decision-maker maLea use of such rtlationships when he 
chooses between conffictin; objectives. lie does cot make his value dectaon 
in a varuiffl. It is further argued that the shape of the curve greatly 
infiuenee his decision. Suppose that in the case of the solid eurv'e he gooses 
the compromise designated by the solid dot which ensures him of the level 
a for 0,. In the case of the lower curv-e he might very well settle for a 
much lower value of 0|, ^ce the value a must be paid for with a much higher 
loss in tenu of 0* than is the case for the upper curve. This bciog the 
case, it seems essential that the dcciaon.inaker u.=e the best possible curve. 
The best possible curve obviously is the one which for each given value of 
0, maximi zes Oj. hlaximization is, of course, with respect to all degrees 
of freedom left within the r^tnetioDS of the prohfem. This particular curve 
we call the ‘efficient curve.' In the case of more objectives, an 'efficient sur- 
face' can be defined in completely analogous fashion by zsaximizisg one given 
objective for fixed levels of all others. For objective functions with rather 
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general convexity properties, the efficient surface also has the stronger prop, 
erty of nwximjzing each obieettvc for fad Icfvls of all cthere. 

To understand liow tins procedure works let us consider an example 
in detail: the simple lot^size problem. This problem consists of 
finding the value of the dectsiOR variable 

g ■» the number of units to be produced in a production run 

which will minimize the sum of two costs: (1) the inventory invesl- 
raent per year and (2) the cost of setUng up erjuipment for a produc- 
tion run per year. In effect, then, there arc two objectives; (1) to 
minimize average inventory, and (2) to minimize tlie number of set- 
ups per year. To m.ako these objectives commensrjrsto (i.e., to pro- 
vide the neee«sarj* transformation) the dollar scale is ordinarily used 
as a standard. Then tiic coat patamelera arc 

Cl - the cost of holding ono unit In inventory per year 
And 

Cj » the setup cost per production run. 

The item parameter is 

R •* required number of units per year. 

Tlie ovteome variahJes arc 



yi average inventory 

(39) 

and 

1/2 = — “ the numl^cr of runs per j'ear. 

1 



(JO) 

The total 

cost, then, is given bj' 



o R 

TC “ CiVi + Czj/a ” 2 

(41) 


Note that the two objectives (to minimize average inventory and to 
minimize the number of eelops) have been transformed into a com- 
mon (dollar) scale by use of Cj and C». To find the value of q which 
minimizes the total cost, wc proceed as follows: 
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Solving. \\c get 



Now, if R, Cl, and Cj were known, <]» and TC, could be determined. 
But suppose that we do not want to transform the second objective 
(to minimitc setups) into a cost yet. Then, according to the a pot- 
teriori procedure, we can treat Cz as a variable and find the values 
of TC» tor various values of Cz. For example, ii R ~ 200 and 
Cl sa $25, we would get the "ctEcient" curve shown In Figure 3.4. 

The decision maker is then asked to select the point at which he 
wants to operate, and the corresponding value of 9 * is used. 

Bebospeetivo optunlzstioa 

A weaker procedure than that described in the use of efficient curve* 
may be employed when ( 0 ) the Ume necessary to calculate the effi- 
cient surface is sot available, and (b) past decisions of the type under 
study are available for examination. TTie procedure consists of “using 
the model in reverse" on past decision* of the kind under study. Past 
decisions are assumed to have been optimaL Then by use 0 ! the de- 
cision model (to be discussed in the next chapter) a determination is 
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raade bs to n-hst tranfformstioa of values r-ouW ba^’c been neces- 
sarj* to have made the past decisions optimal. This procedure is re- 
peated for several past decisions, and Uje results arc compared. If 
(hey have been relatively consistent, then t!ie researchers have a 
general notion of the location of the equal trade-off point. If the 
results are not consistent, the decision maker may judge which past 
decision was best and the trade-off associated with this decision can 
be used. 

In a study involring a public utility the question was raised con- 
cerning the number of service garages from which men were dis- 
patched to answer calls from customers whose electricity was "cut 
off." The problem int-olved two familiar objectives; minimUation 
of cost of operation and minimisaUon of service delays to the cus- 
tomer. The cost of delays to the customer could not be obtained by 
cither objective or SJtliicclive analyses. The procedure for evaluating 
a course of action involved a complex and tlme-coasmaing simulstion 
which could not bo repeated for a variety of assumed delay policies. 
TIie time allotted for the research prevented making such compu- 
tations. 

In the past, however, the company had made two decisions of the 
kind under study. These were assumed to be optimal, and a deter- 
mination was made os to what range of values in dollars of an aver- 
age miaute'a delay to the customer made these past decisions optimal. 
The ranges obtained from the two past decisions were almost identi- 
cal. Consultation with roanagcincnt disclosed no basis for question- 
ing the reasonableness of the results. The average raldninge was 
then used to find the "best" decision in tljc current decision situation. 

Retrospective optimisation is obviously the weakest of the trans- 
formation procedures described. It assumes that the decision maker 
has performed oprimaHy in the past. Tim obvious question is, if this 
is assumed, why not let him moke Uiia decision in the same way he 
has decided previously? The principal advantages of this procedure 
are that it makes consisfent decisions (though perhaps inefficient ones) 
more likely in ilie future and that it raises past implicit transforma- 
tions to consciousness, where they are likely to receive more careful 
consideration in the future. 

Effectiveness curves 

Suppose that in the example used ia connection with efficiency curves 
the researcher is not only uBwilling or unable to express Cj, the setup 
cost per production run, but also unwilling or unable to do so for Ci, 
the unit Inventory-holding cost per >^ar. He feels, however, that 
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the ratios C 1 /C 2 and Cs/Cj are nevertheless meaningful to the de- 
cision maker. lie could then follow a procedure described by Feeney 
in 1955. 

The optimal values of the outcome variables, yi and ys, may be ex- 
pressed as follows: 


Vu = 


y3. = 


vmc,/c,) 


il" 


(45) 


?a V2IHCi/Ct) 


M- 


(46) 


'Ilien 

ICt fH j^, [n R 

l/uyj. = ^01^2 ■VCj "Vi * 2’ 


(47) 


Assuming R =« 200 as before, we can now plot the effeclivenets curve 
shown in Figure 3.5. 

Feeney (1955, p. 75) offers the following explanation of the effective- 
ness curve: * 

Any point of the cun-c is attaiaable through the decision rule ... de- 
rived and u efficient in the eense that the outcomes associated nith that 
point )-ield a minimum total variable cost for some cost parameter ratio 
[Cj/Cj]. Thus any relalionslup between inveatoiy investment and [number 
of setups per year] that lies on this curve may be attained and will be the 
lowest cost rclationriup for some set of cost parameters. Conversely, for 
any given cost pararactcra used in the decision rules, the resulting outcomes 
must he at a point on this [cffecUvcness curve]. 

lie goes on (p. 77) to evaluate this procedure as follows: 

Tliis mechaniam is the mind of the decision maker himself. We conclude, 
therefore, that in situations in which it is not possible by direct methods to 
obtain meaningful measures of the cost parameters contained in the decision 
rules the [final] decision cugbt best be nude by confronting the [decision 
maker] with a picture of the [effectiveness curve]. In essence, we present 
him with the set of all poeaMe alternatives that have the property of being 
optima! under some set of costs. If, as b sescrally the case, the [effective- 
ness airve] b convex, we may make a >try strong statement with re- 
spect to the properties of these alternatives, namely, that so long as total 
variilite cost b non-decreasing over the range of each of the outcome van- 

•The brarVclnl ciprcssions are substitalea for Feeney’* terminology, with 
which that UKd here baa not consent. 



FOBiHTLATlNO TOT PBOBLEM 103 



obles ibe [effeetivens^s cune] derived iiotler a linear cost model must coti> 
(Ain the optimal outcome regardtess of the nature of (lie cost function that 
is aclaiiiy employed by tlie decision maker. That u, even if the cost func- 
tion is non-linear and (lie outcome variabtea themselves are subject to a 
muftiiuifo of complex restrictions, some pmnt on the [cfTcctivcncss curve] 
must provide miiunmm total variable cost. 

Returning to the simple lot sue formulation that i\aa explored earlier, the 
curve shown in Figure (w some inoditted form of this cun-e) might be 
presented to the executive lie rould be told tiuit under (he rules that have 
been developed any relationship bctweeri inventory [average inventory] and 
[number of setups] jicr year that lies on this curve can be attained and that 
re^rdlc.'s of the fsclors that arc actually taken into account, some point 
on this curve must be the best available outcome for the company. lie 
would be told also tlmt the final wlecUon of a point on this cunc as a target 
for company inventoo" operations is a deddon that be himsid/ must nwke. 
This picture might bo elaborate*! throi^li such things ns an indication of 
the outcome attained Ihrougb caasting controls to provide some perspective, 
and jierhapa by an indication of the point or points along the cune tfwt 
would be selected through a cost ncconnUne study of the underlying cost 
parameters. We believe through the use of such an appro.ich the analyst 
provides the best available basis for executive decision making and avoids 
the und^rablo practice of working bQ-ond the limits of hli methodology. 

The a poslerifin methods just deseribed have a great deal of appeal 
to rr.’enrcbera because they appear to be relieved of the gcnemlly 
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difficult tasks of finding transformation and value functions. They 
are so relieved, but at a price It is this price that we want to ex- 
amine DOW. 

If any of the variables in a problem situation and its model are 
probabilistic, then the outcome is also probabilistic. This is handled 
in the a posteriori method b>' dealing with expected values rather 
than distributions of values, and licncc deterministically. Conse- 
quently, even if the point chosen by the decision maker maximizes 
expect^ effectiveness (or minimizes expected ineffectiveness), it will 
maximize (minimize) expected value (loss of value) only if the value 
function of the scale involved is linear and monotonically increasing 
in the case of maximization and monotonically decreasing in the case 
of minimization problems. Therefore, determination of a value func- 
tion is avoided by assumptions, assumptions which seldom turn out to 
be valid where value functions are explicitly derived. For example, 
because of the threat of monopolistic charges by tlie government the 
value of increasing share of the market for most companies is not 
monotonically increasing. The infrequency of linear-value functions 
has already been referred to. 

If the problem is truly determiDlstic, then the linearity assumption 
concerning values is not required, but the assumption of monoto- 
nieUy is. 

If three objectives are involved, the efficiency and effectiveness 
surfaces can still be represented in two dimensions. An example of 
such a surface, taken from Feeney (1955), is shown in Figure 3.6. If* 
however, more than three objectives arc involved, graphic representa- 
tion is virtually impossible and the decision maker must be con- 
fronted with extensive tabulated data the significance of which is 
almost impossible for him to understand. The evaluations asked of 
him under these circumstances are considerably more difficult to make 
than those involved in obtaining explicit subjective transformation 
functions or value functions. 

Finally, the computational requirements for the a posferion method 
where three or more objectives arc involved become very great. Use 
of a high-speed computer is almost essential. Although this may not 
be prohibitive, the alternative use of time to find the transformation 
and value functions explicitly should be carefully considered. 

Despite the shortcomings of tbe a poslertori method the researcher 
may be forced to use it, as Feeney has observed. If he does so, he 
should be aware of the assumptions involved and these should be ex- 
plicitly formulated. The dewion maker should be informed com- 
pletely of them and their significance before he is asked to reach any 
decision. 
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A SATISnClNC rnOCEDURE 

In some situations it nay not be pracUcally feasible to use either 
the a jrrion or a poslerion method. For example, limitation of re- 
sources for the research (c g., time or nioney) may preclude the neces- 
sary effort to perform the rcquiml transformations, evaluations, or 
computations. In such cases tlie foiloiving procedure is commonly 
used. The most important objective (judged in some qualitative way) 
is selected as the basis for the measure of performance of a course of 
action. Minimal le^-els of acceptable performance (subjectively de- 
termined) are imposed as restrictions on an acceptable solution. Such 
a level may, for c-vample, be (he level of performanee attained in the 
past. 

Consider a case in which the detn^on zaalcer wants to reduce costs 
and improve service. Suppose that a traasfonnation or evaluation ia 
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precluded on practical grounds. The decision maker may be willing 
to settle for this criterion of perfonnance: to minimize costs subject 
to the condition that the quality of service docs not deteriorate below 
the current le%’el. If service is improved, fine; this is n bonus. But 
it will not be permitted to get worse. 

Such a procedure combines the principles of optimization and satis- 
ficing; optimizing with respect to costs, satisficing with respect to 
service. If alternative optima relative to costa are found, that action 
which docs beat relative to service is selected. 

The principle involved here consists of reducing all objectives but 
one to a constraint, an objective to be retained or a qualitative ob- 
jective that must bo attained. Since optimization is not involved here, 
satisficing is; that is, doing at least as well ns some specified level of 
performance relative to the objectives involved. Then optimization 
takes place with respect to only one of the objectives. This proce- 
dure eliminates the need for transformations, but not necessarily for 
evaluation of tlic scale used in conneclJos with Die eptimizalion ob- 
jective. In order to select among the efficiency functions for alterna- 
tive courses of action it may be necessary to know at least some char- 
aeteristles of the value function of the scale (c.g., if it is roonotonically 
increasing). 

The satisficing procedure Is not to be recommended except where the 
a priori and a posteriori procedures cannot be carried out. It embod- 
ies very strong assumptions concerning the value functions of the al- 
ternatives. It assumes that performance better than the acceptable 
level relative to the objective to be satisficed has negligible value rela- 
tive to improved performance on the scale with respect to which op- 
timization is performed. 

It, of course, an action can be found which performs best with re- 
spect to both scales, it is the best course of action regardless of the 
transformations and evaluations which might be performed. Such a 
uniformly best alternative is not hkety to be found from among a 
large set. The existence of sneh an altemalive depends entirely on 
the consistency of the decision maker’s objectives. But roost problems 
are characterized by some incompaUbility (conflict) between ob- 
jectives. 
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MODELS 


The word tnodel is used as a noun, adjective, and verb, and in each 
instance it has a sUgiUly diScrent connotation. As a noun “model 
is a reprtuntalion la the sense in which an arcliitect constructs n 
smalbscale model of a building or a physicist a large-scale model of 
an atom. As an adjective “model” implies a degree of perfection or 
idealization, as in reference to a model home, o model student, or a 
model husband. As a verb “to model” means to de>nonitrate, to re- 
veal, to show wbat a thing is like. 

ScientiEc models have all these connotaUons. They arc representa- 
tions of states, objects, and events. They are idealised in the sense 
that they are less complicated than reality and hence easier to use for 
research purposes. These models are easier to manipulate and "carry 
about" than the real thing. The stmpUcUy of models, compared with 
reality, lies in the fact that only the relevant properties of reality arc 
represented. For example, in a road map, which is a model of a por- 
tion of the earth's surface, vegetation ia not shown, since it is not rele- 
vant with respect to the use of the map. In o model of a portion of 
the solar system the balls representing planets need not be made of 
the same material or have the same temperature as the planets them- 
selves. 

Scientific models are utilized to accumulate and relate the knowl- 
edge we have about different aspects of reality. They are used to 

108 
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reveal reality and — more than this — to serve as instruments for ex- 
plaining tlic past onil tJtc present, and for predicting and controlJicg 
the future. What control science gives us over reality we normally 
obtain by application of models. They are our descriptions and ex- 
planations of reality. A scientific model is, in effect, one or a set of 
statements about reality. These statements may be factual, law-like, 
or theoretical, 

In science, as well as in ordinary activity, we employ different types 
of models; the iconic model, the analoffue, and the iymbolic model. 

Iconic models arc large- or small-scale representations of states, ob- 
ject®, or events. Because tbej' represent the relevant properties of 
tlie real thing by those properties themselvs, with only a transforma- 
tion in scale, iconic models look like what they represent. For exam- 
ple, road maps and aerial photographs represent distances between 
and relative positions of places and routes between them. IFitii re- 
spect to these relevant properties such maps or photographs look like 
the real thing; they differ from it with respect to these properties only 
in scale. Flow charts which show the processing of material or in- 
formation may also be iconic models, as may be floor plans or otlicr 
types of diagram. 

In most cases if wc want to show relief (i.e., the third dimension, 
elevation) on a map we do not produce a three-dimensional map; 
rather we resort to colors or to contour lines which by their distance 
apart convey information about grades. Or, if we want to show the 
kind of road, we use color or shading and provide an appropriate 
legend wbicli explains the transformation of properties. la these 
eases, one property is used to represent another, and hence the neces- 
sity of a legend. In such cases the model is an anatoffue. 

An electrical system may be represented by a hydraulic system. 
In such a case the flow of water may represent the flow of electrical 
current. The slide rule rs a familiar analogue in which quantities are 
represented by distances proportionate to their logarithm. Graphs in 
which such properties as costs, time, numbers of people, and per- 
centages are plotted are also analogues. 

Finally, there are symbolic models in which tlic properties of the 
tiling represented arc expressed symbolically. Thus, a relationship 
shown in a graph (as an analogue) can also be shown in an equation. 
The equation is a symbolic model. Models in which the symbols em- 
ployed represent quantities arc usually called mathemaltcal models. 

Iconic models are the most spedfic and concrete of the three types 
of model but are usually the most difficult to m.anipul3le for purposes 
of determining the effect of changes on the real thing. In the analogue 
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casicr-to-manipulate propcrtira are usually subsUlutctl for the real 
ones. As a consctjucnce such models are more abstract and general. 
For example, U we examine a graph in which distance and time are 
plotted, it is not likely that we ean identj/y the phenomena involved 
unless the graph is labeled appropriately. SjTnbolic models are the 
most abstract and general and arc the easiest to manipulate. In gen- 
eral, the amount of analysis require*! to construct a model is inversely 
related to the ease of manipulating it once it lia-s been constructed. 

Science employs all three types of model. In general, however, it 
ii«cs iconic models and analogues ns prclimin.arj’ (o (he development 
of sjTnboVic models. Iconic moflcls and analogues arc also uscl widely 
by Bcicntista for pedagogical purposes, since they are easier to un- 
ilcrsland. 


VAIUADLES 

Mathematical models in science usually lake the form of equations 
( = ) or inequations (>, <, ^). On the left side of the equality 

or inequality sign we normally place the variables who«c past values 
we arc trying to explain or whose future values we want to predict 
or manipulate. These variables may be thought of as describing an 
outcome or output of a process. Tiie \*ariablfs on which these values 
depend are placed on the right side of the equality. These may be 
thought of as jnjnds. 

In the familiar gravitational model 

s = ijt* 

8 (the distance travelcti by a freely falling body) is tlie variable to 
be estimated, g is the gravitational constant, and ( is the number of 
seconds elapsed from the start of the falL By an algebraic transfor- 
mation we can get 

t = V2»/g, 

By such transformations we can convert the role of a variable in a 
model. We do not obtain a complete description bj’ naming the de- 
pendent and independent variables; a description of their relationship 
is also required. We can write 


* = Ha, 0. 



UOVELS 111 

ivliJch saj's that « is a function of ff and i, hut / must be specified. 
Tills is done by the appropriate insertion of and the square. 

The gravitational equation is a model of a class of phenomena but 
is not a model of a problem situation. 


MODELS OF pnOBLE^^ SITUATIONS 

Models of problem situation*, as we have discussed tliera, should 
always talce the following special form: 

r=/(A',,iv), ( 1 ) 

where V = the measure of the value of the decision tJiat a made (i.c., 
the action taken). 

Xi «a the variables which are subject to control by the decision; 

the decision voridWes define the ollcmativc courses of action, 
ly • the tacton (variable or constant) n-bich affect perforraancs 
but wliich ore not subject to control by the decision maker 
within the scope of the problem as defined ; these we shall call 
'‘parameters," 

/•a the functional relationship between the inclepeiidcnt varia- 
bles and constants, A‘f and l*y,ond the depeudent I’ariablo T. 

A model of a problem situation has two essential characteristics. 
First, at least one of the “input” variables is subject to control by the 
penonfs) confronted by the problem (i.e., it must model hia possible 
choices of action). Second, the “output” variable must be a measure 
or index of the value of the nUcrnalivc choices to tlie decision maker. 
Models which satisfy these two conditions may be called dcCTrion or 
problem models. 

In some problem situations the values of the decision variables are 
constrained or limited. For example, in problems im*olving the alio- 
cation of resources, the amount of resources that can be allocated 
cannot c.vcced the amount avaifable. The total amount of time a per- 
son can allocate to various tsrpes of activity cannot exceed the amount 
of time available to him. If U is the amount of time allocated to the 
ith activity, then the constraint can be represented bj' the inequation 

wliere T is the total amount of time available. More than one con- 
straint may exist, and consequently more than one equation and/or 
inequation may be required as part of the model. The decision model, 
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Chen, ffiay consist of a set of equations and inequations, the core of 
which takes the prototype fonn 

r = /(.Y., r,). 

It should also be observed that in some problem situations the 
choice available to the decision maker cannot be represented by a 
quantitative variable; rather the choice is between discrete qualiU- 
tive alternatives. For example, which of several brands of an in- 
strument should he used in a control ej'stem? In such a case the 
available brands cannot be conveniently represented as a quantitative 
variable. In these cases, then, the model may be expressed as a set 
of equations of the form 


r, = f(Xu y,) 


1 ',) 


( 2 ) 


where ..Yi, Xi, ■ ■ • represent the discrete elioices, and Vt, Vs, J’SP" 
resent their performances. 

Decision models may be applicable to the decision maker’s problem 
or to decisions which the researcher must make either in trying to 
find a solution to the decision maker’s problem or in answering a 
question. We shall call the latter reaeorcA models. "Optimising re- 
search decisions," then, requires the construction and solution of ap- 
propriate research models. We are not yet capable of constructing 
all the relevant models, nor are we capable of sohdng all the models 
we can formulate. In this book wc will tiy to go aa far as possible 
in the construction and solution of such models. 

The general discussion provided here about decision models is 
equally applicable to the special type of decision model, research 
models. 

In the construction and use of decision models many other models 
may be required. In some cases wc incorporate other models into a 
decision model, and in other cases we use them as preliminary steps 
in the construction of a decision model. To illustrate the former, 
consider a model of the operation of the toll booths at the entrance 
to a bridge or tunnel. One vanahle Uiat is very likely to appear is 
the "average waiting time of a motorist ” To determine what thi- 
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average uniting time is, U may be necessary to construct a model 
uliich expresses tliis average lime as a miitlieinatical function of the 
number of toll booths in operation, the service rate of each booth, and 
the rate of customer arrivals. This waiting-time submode] can then 
be substituted for Uie waiting-time variable in the more general model. 

Afodefs as a Source of Control 

A model wliich can be used lo predict a future outeoinc may not be 
suitable for use in controlling that outcome. It may, however, be 
useful in developing a model wbich permits control. For example, 
consider the following digest of an actual research project. An oil 
company wants to determine where to locate its service stations and 
what types of service stations to build so as to maximise its sales 
volume.* The researchers found that no adetiuatc theory exists to ex- 
plain the volume of business that a service station obtains. Tlicy 
drew up a long list of factora which may affect sales vofurae. The 
most important of these seemed to be the volume of traffic that passes 
the service station. The researchers decided to concentrate initially 
on determining the extent of the effect of traffic on sales volume. They 
selceted a number of the company’s service stations which are located 
at intersections of two streets meeting at right angles, at which Inter- 
sections no other service stations are located. (The number of adja- 
cent stations is another variable that was investigated subsequently.) 
It was observed that traffic can come into the intersection from four 
different directions and can leave the intersection in any of four 
directions, assuming that a tum-arouod is possible, as it is through 
a service station. Consequently, there are sixteen different routes 
through this type of intersection. Data were collected on the number 
of cars passing through the intersection on each route and on tlic 
jwreeatage of cars in each roule that stopped at the station. These 
data were tabulated and analysed, and it was found that the percent- 
age of cars from each route which stop at the station varies signiG- 
c-antly. By regression analysis an equation was constructed that pre- 
dicted the volume of sales aa a function of the average number of 
ears traveling in each route per unit time. These predictions were 
not perfect, of course, because many variables wore omitted. But it 
turned out that tl)e predictions obtained from the regression cqu-ition 
were ccusiderably better than those obtained from the predicting pro- 
cedure then in use by the company, even ti»ough the latter took into 

'Tlie objective Is deliberiUty Amplified in Ibis dtseuadon. 
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decision — we normally begin by listing all tfie variables that we believe 
might be causally related to the outcome. In such cases it is better 
to include doubtful variables than to exclude tliem because — as we 
shall see — it is subsequently much easier to identify and delete an 
irrelevant variable than to identify and insert a relevant one. 

Wiiat we do next depends on whether or not we can manipulate the 
variables listed. If we can bring the phenomena into the laboratorj' 
or can control them in their natural setting, we can then conduct ex- 
periments to determine which variables are causally related to the 
outcome and bow. If such control is not possible or practical, we 
must restrict ourselves to the data ax'ailablc from the “natural" (i.e., 
in context) behavior of the plienomcna and we must usually resort to 
the mctljods of regression and correlation analysis or variations 
thereof. 

In the service-station case referred to above, initially there was 
no acceptable explanation of sales volume, and one had to be devel- 
oped in order to determine which properties of a service station should 
he manipulated so as to aflfect its sales, and liow to manipulate them. 
InUially, then, ail variables (recognised by those involved in the 
problem) which might aSect sales were listed. Each variable had to 
be defined is such a tvay that the observations required to test its 
relevance were specified. Such deCnitions had to specify how meas- 
ures {quantitative observations) of the variable were to be made. 
The service stations to be obscrs'cd Jiad to be speclbed fi.c , the popii' 
lalieK defined) and a decision made as to how many and which sta- 
tions to observe (the samplatg design). The procedure for making 
the observations, recording the results, tabulating, and analyiing 
them also had to be designed. Id subsequent chapters we will con- 
stder each of these phases of tlie research in detail. Tliat is, we will 
consider defining, measurement, sampling, estimation, and procedures 
for analyzing these data. 

In the case in which we can control the values of tlio “candidste" 
variables (i.e., those in question) these same procedures arc involved; 
but rather than sample elements, situations, or events under natural 
conditions, we can specify the conditions under which they arc to be 
obser\’ed and either how these conditions are to be constructed or 
flow they are to be found, if they exist Here the problem is one of 
erperimentttl design, to which subject Chapter 10 will also be devoted. 

Sometimes an explanatory model can be constructed without first 
cither developing a descriptive model or conducting experiments. 
These arc cases in which a different type of observation and analysis 
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of the phenomenon involved taji !ca«l one ilireclly to nn cxplanatorj’ 
model. Required for such eases is a certain nmount of prior infor- 
mation about the procest and—rclativc to our Btutc of knowledge — 
a rertnin fiinjOieity iif tlic phenoincnoo involved. 

Consider the problem of a new’sboy who must decide how many 
neo'spapere to order each day. He makes a certain amount of profit 
on each newspaper he sella and incurs a certain lo«3 on each paper he 
returns. Ilis losses and gains depend on how many newspapers lie 
sells. This quantity varies from day to day, but is predictable in 
the sense that past data provide estimates of the probability that any 
specified number of papers will sold on any given day. 

We can begin the construction of a model of this problem situation 
by listing the relevant factors and assigning a sy’mbol to each: 

n « the number of newspapers ordered per day. 
a ■ the profit made on each newspaper sold, 
b - the loss incurred on each newspaper that is returned, 
r - the number of requeals from customers for newsp.apcrs. 

P “ net profit per day. 

If P is negative, the newsboy has su«taine«l a loss. On any particular 
(lay his profit can be cxpresseil as 

P » ro ~ (n - r)l», if he orders more than ore requested, and 
P = na, i( he oixlers less than are requ(*s(pd. 

To determine how many newspapers tfie boy shoutd order we must 
consider more than one day; that is, the average effect of his decision. 
In order to do so ne must introduec the following concepts: 

P(0 = the probability that r requests will be made on a day- 
P = the average net profit per day. 

Now we can construct n general decision model for the satuation; 


^ = 2 pWf''® - (« - r)6J + 2 P(r)-na. (3) 

'-0 r.n+l 

In this model F is the measure of performance, n is the decision variable, 
r is the uncontrolled variably and a and b are uncontrolled constants. 
The problem, in this case, is to find the value of n which maximizes T‘ ■ 
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In tlie illustration llie sinicturc of the problem situation is rather 
apparent, and hence a causa] model can be obtained directly from a 
description of the situation. But this may not bo the case even for a 
situation tliat can be described simply. For example, an industrial 
sales manager may want to divide his available funds between ad- 
vertising and selling effort so as to maximize sales volume. Since the 
relationship between sales and advertising, for example, is not known, 
Ksearch woukl first Imve to Ik conducted to uneoivr this relationship. 


MODELS AS APPROXIMATIONS 

All models of problem situations arc approximate representations of 
these situations. They are generally simpler than the situations they 
represent, which arc usually so complex — although in many cases, 
fortunately, only* in unimportant detail — that an "exact" representa- 
tion (even if possible) n ould lead to hopeless mathematical complex- 
ity. Therefore, the protiletu confronting the model builder is to at- 
tain a best or good balance between accurate representation and 
mathematical manageability. Pr.aetical considerations (e.g., limited 
time, money, personnel, or computing facilities) almost always le- 
(]U)re some compromise of accuracy. 

Tlie mathematical manageability of concern here la of two types: 
one involving the researcher, and Utc otlier t!>e decision maker. On 
the one hand, construction of n model that the researclier cannot solve 
because of mathematical complexity is of little value. On the other 
hand, the derivation of a solution which is too complex mathematically 
for the decision maker to use is also of little value. 

The simplifying assumptions on which model approximations are 
based should be made explicit so that U>e researclier can determine 
in wbat direction they falsify the problem situation and by how 
much. • Such a determination is essential in the justification for the 
approximation. 

Complete justification of an approximation, however, requires a 
comparison between the "cost” arising from mathematical complexity 
and the cost of lost perfonnance of tfie course of action selected by 
use of the model as compared with a less approximate model. The 
cost of the mathematical complexity includes not only the cost to the 
researcher arising out of computational diSculty but possibly also the 
cost to the decision maker in appijring the solution if he must do so 
himself, ns may be the case in repetitive decision making. 
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Tlicsc approximation co'Ls are likely to l)e more cliiBcuU to deter- 
mine in practice than to in principle. Tlic complete coit of 

approximation would rcfiuirc an "exact" iiioilcl with whicli to com- 
pare the approximation Tlic exact ino<!cl is ecldotn if ever available, 
lienee, the calculation nf costs in practice, if at all po««iltte, must I>e 
bascii on a compari'on of two models, botli approximate but to dif- 
ferent degree?. Differences in computational and analytic costs gen- 
erally have to be determined by trying the alternatives. Determine* 
tion of costs due to loss of performance can al'O l>e determined either 
exactly or approxim.atcty by analyids or computation. In the ex- 
amples which follow such detennin-ations will Ijc considered. 
Approximations ore used in tw© ways in model conslrucUon; 

(1) To "sneak up" on the prohicni I>y successively more complex 
appro.ximation3. TJ»s procerJure is likely to be U'ed when the 
searcher cannot initially grasp all the complexity of the real situation- 
For example, in discussing hU approach to modeling the operoUon of 
complex systems Glen D. Camp (1057-58, p. 10) said: 

I prefer to start wiUi two or a iwmll nuishcr of the crudest and eimplesl 
models which appear to offer any hope of reprrfcnting Uie major features 
of the operation under study, eboc^g them so as to "surround" the real 
operation as well as possible (that is, attempting to falsify the two or mere 
models in opposite directions so as to bound the real operation). These 
models can usually he "operated" by simple desk calculations, the results 
being lucd to judge the desjrabflity of further refineracnts and the directions 
nhieb thcpe should take if deemed desirable. 

(2) To move down from a model Uiat b loo complex to handle to 
successively simpler approximations until one is obtained that is both 
m.anagcaWe and accurate enough. This proenfure will be illustrated 
in the case (which will be discussed in the next section) involving 
production of a chemical. 

The researcher who uses the second of these procedures (complex 
to simple) has an explicit standard against which to judge the sim- 
plicity of successive models. In the first procedure (simple to com- 
plex), judgments of simplicity are based on an implicitly formulated 
complex model. This docs not mean that the second procedure is 
better. tVhether one should approximate “up” or “down" is largely a 
matter of personal preference and the complexity of the problems at 
hand. Ideally, the net result should be the same. 

Approximations can be applied to any of the componenU of a 
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model: the variables, relations, and limitations or constraints. We 
will consider each of these in turn. 

Omitting Decision Variables 

Variables known to he relevant may be deJiberately omitted, gen- 
erally because their impact on performance is small and their con- 
trihulion to matliematical complexity is large. This is equally true 
of decision (controlled) and uncontrolled variables. 

Omission of decision variables is illustrated in a model which was 
developed to control the production of a biochemical. The process 
yields three products which wc shall refer to as "Crude Q," "Fmished 
Q,” and "A.” Crude Q and Finished Q are products for human con- 
sumption; the former is made in bulk and semifinished form. Jl is 
made for animal consumption. The production process involves ap- 
proximately fifty distinct chemical operations. This number inado it 
impractical (from both the accounting and the mathematical point 
0 / view) to construct a cost equation for the process in which the cost 
of each operation was treated separately. It was decided, therefore, 
(o construct a less comple.x diagram to describe the process, a dia- 
grammatic analysis which grouped operations around the points at 
which the process h controlled. This diagram (Figure 4.1) was also 
designed to group the operations so as to permit acquisition of cost 
data in a convenient way and yet show ail possible channels of pro- 
duction, 

The discs in Figure 4.1 indicate points at which there is a division 
of material on the basis of its usability. At such separation points, 
some material may be suitable only for the R processes, some may be 
suitable for both the Q and the R processes, and some may have to 
be disposed of as waste. Waste, os it is shown on each of the charts, 
includes material sewered (because it is uneconomical to reprocess) 
and process losses resulting from each operation. At each separation 
point in the Q process a portion of the material is suitable for R 
processing, but it requires further processing or recycling to be suit- 
able for Q. The small dark rectangles in Figure 4.1 indicate where 
control decisions regarding the allocation of usable materials to vari- 
ous channels must be made by those in control of the process. Then, 
in addition to tlie decision ns to how many fennentors to set, there 
are six control decisions involved in the production process (shown in 
Figure 4.1 as A through F): 




4 1 Flow dUpam ot n biochemical proceas. 


Control-Decision 

Syrnhol 
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Control Decision 

A Determination of tiiie portion of raateriaLi suitable for both 

Q and It proiJuctibn that is to be sent llirough (he Q proces.‘<. 

n Determination of the portion of the niatoriaU suitable for the 

finished and Chwle Q production ttiat is to be sent to 
the rinished Q productiem. 

C Determinattoa of the portioa of Uie materials suitable for R 

production Uiat is to be sent to special process 4Q] which 
eventually rnabea more mateiist available for Q processing. 

D Determination of the portion of the materials suitable for It 

production or reeyeling that is to be recycled after the 
Crude Q conversion process. 

E Determination of the portion of the materials suitable for ii 

production or recycling that is to be reigrcled after the 
f inish ed (J conversion process. 

F Detenninalion of the portion of the Crude Q inventory that is 

to be recycled through (ha Finished Q conversion process. 

A model of the problem was constructed which expressed the eoat 
of the process as a function of the six decision variables. The result 
was an equation that was much too complex to handle even -with the 
aid of an electronic computer. The principal reason for the complex- 
ity arose out of the “recycling” possibilities; that is, decisions C, V, 
E, And P, at which points the material could be returned for reproc- 
essing. If these controllable variables could be omitted, the model 
would become manageable. Could such omission be justified? 

An analysis was made (o determine how the control decisions af- 
fected the yield of the process. Six extreme cases were defined. In 
the first ease all suitable material is sent to Finished Q at every con- 
trol decisions; the material not suitable at B is sent to Crude Q; and 
what is not suitable at C, D, and E is sent to R processing, and no 
Crude 0 is reprocessed. (That is, A «= 100, £ = 100, C = 0, D » 0, 
E = 0, and F — 0.) This case involves no recycling. This and the 
other five cases are shoum in Table 4.1, os well os the yields of each 
product expresserl as percentages of the fermentation yield (X). 

Table 4.1 shows that the more material recycled the greater is the 
waste. Specifically, a comparison of no recycling with complete re- 
cycling for daily settings of one or two fermentors, after allowance 
for noarecoveraWe terwenioTS has been made, shows that tliere is 6 2 
per cent (38.7 per cent — 32.5 per cent) more fermenter yield avail- 
able for processing from the same amount of raw material. For a 
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Table 4.1, Usable Prodcct ExFUEsaso as a Pebcentaoe op 
Fermentation Yields (X) fob Vathoub Control Dectsions 
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* TUa quantity U the total of the cxpecteU Iobms in the tytlen, co&sisUns of tha 
pioeeu-pluuie looses plus unusable qunoUUes which are sewered. In addition to this 
waste, there is also on expected waste of O.OSl ferntentor per day at the fermentation 


t Since the number of fermentots that can W sent to Q proceesing per day is 
Unuted by the nature of the process. If the maximum number of d.xUy Beltings is made, 
A can be no greater than 73 8 per cent 


maximum daily setling of femientors, 7.0 per cent (36.7 per cent 
- 29.7 per cent) more fermentor yield is available. 

The fact that recycling decreas^ yield xvaa known to management. 
But the amount of this decrease was not known. A study was then 
made to determine how the reduction of waste through the elimination 
of recycling would affect total production costs and income. The re- 
sults are shown in Table 4.2. Tlie income was computed by assuming 
that all material produced is sold at the average market prices. 

Tiie difference between income and process cost is defined as a re- 
lated net income. (Note that tlib is not a net profit ) 

Table 4.2 shows that, compared with complete recycling, produc- 
tion without recycling yields twice as much related net income for 
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rainimuni fcrmentor tcUings, and one nod one-third tiinee as much 
for tnnximiim fennentor EctUngs. 

That production economy results frojn eliminating rwyel'ns 
firmly substantiated by these results. It was then necessary to de- 
termine whether sates rerjuirements could be met if recycling were 
eliminated. 

A eompari'on of ntiuunl process capacity with shipments for lOM, 
1031, nnd 1032 was made. The results are shown in Table -1.3, where 
1950 shijunents for Fini«h<Ml Q are e»jualo«l to 1 and arc used ns n 
bn®c. All other fipires arc cxprc^cd as multiples of this figure. 

Tabic 4.3 shows that cnougli of cacti proifitct could have been pro- 
duced to meet nnnii.al sales rc<)mremenLs in 1950, 1951, and 1052. This 
is not eoneltisive, howcier, for it docs not show whether siiIcb require- 
ments could iiavc been met on a day-to-efay basis. To determine 
whether short-run requirements could be met, and to determine the 
cost of BO doing, it w.vs neee?«.ar)’ to develop a detailed proceilure for 
controlling the prwfuelion process without rccycting. This was done, 
hut the model Involved was hasnl on the simplified conception ot the 
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Table 4^. Maxisium Ajwdal Yoids witbodt Recycunq Compared 
WITH Annual SHiPUEsra 
Annual Yield 

A - 0.73S Annual Shipments 

Product B-0.5 B=.a95 1050 1051 1952 


Finished Q 3 6 7.4 

Crude Q 56 1.2 

R 16.6 16.4 

Total 25 8 250 


process (shoTcn io Figure 4 2) which resulted from the elimination of 
the four decision variables involving recycling. 

Omitting Uncontrolled Variables 

Omission of uncontrolled variables is illustrated by the usual model 
employed to determine ‘'economic production quantities” which was 
developed in equation <41) in Chapter 3: 

TC-C.^ + Cj- C-1) 

2 q 

where TC *=■ total incremental cost per year, where “incremental 
refers only to costs affect^ by the production quantity. 

Cj = cost of holding one unit in invenlorj- per year. 

q t= number of units to be produced in a production run. 

Ci “ the setup cost per production run. 

R = rcqiured number of units per year. 

This model omits in-process inventory cost; that is, those costs in- 
volved in carrying the investment of money in labor and material m 
the parts being produced. The larger the production quantity, the 
longer is the producUon time and hence the larger is the in-process m* 
ventory cost. An accurate model of this cost is generally very com- 
plicated and, if added to the eimple equation (4), frequently makes it 
unmanageable. In mo«t cases, however, even tripling the production 
quantity will result only in an increase of the in-proecss inventoiy 
cost that is a small fraction of 1 per cent of the total cost of the part. 
Costs can seldom be estimated with an accuracy that justifies (on 
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practical grounds) inclusion of this email in-proccss inventopf cost 
Omission of this cost from the model, however, should be justified by 
a determination of its magnitude. 

The aggregation of several variables docs not exactly omit any of 
the variables, but it does reduce the number that have to be consid- 
ered. For example, in some inventory-supply systems hundreds of 
thousands of items ore involved. It is not feasible to determine the 
best stocking quantity for each of these separately, even if all 
quired models could be constructed. The parts ore sometimes grouped 
into four classes: 

(а) Hi^i usage, high cost. 

(б) High usage, low cost- 

(c) Low usage, htgli cost. 

(d) Low usage, low cost 

In most inventorj’ situations only a small percentage of the items 
carried falls in class (a), but Uicse items generally account for roost 
of the investment in inventory. Therefore, there b generally good 
economic reason for treating these items separately and for using 
average characteristics of items in the other classes. An estimate of 
the error introduced by the "a>'eragiog” approximation can be made. 

In some cases where usage and cost do not vary widely over different 
items the principle of "essentiality" b used to classify them. For w- 
. -ample, in determining a policy for stocking replacement parts for dis- 
continued modeb of heavy industrial equipment the following classifi- 
cation was used: 

(a) Parts without wliich the equipment could not operate at ail. 

(b) Parts without which the equipment could perform some but not 
all its functions. 

(c) Parts without which the equipment could perform all its func- 
tions but only at inconvenience to the operator or at lower efficiency 

(d) Parts without which the operation of the machine was unaf- 
fected. 

Modeb are then prepared for only these classes, again using average 
values for each class. 

Where variables are aggregated, the error (in the estimate of the 
outcome) which b introduced is roughly proportional to the ratio of 
the within-aggregation variance to the between-aggregation vari- 
ance. Put another way, it is desirable to make the variables aggre- 
gated, as homogeneous as poarible and the aggregations as heterogene- 
ous as possible. 
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Glen Camp (1957-58, p. 12) discusses this principle of aggregation 
and illustrates it as follows: 

The labor of solving pToblems toereasea roughly eiponentiaHy with the 
number of variables, an<l hence it is important to begin with as few variables 
as seems likely to yield a useful represmtation. To this end, we often ag- 
gregate varisbias and it is therefore important to be able to estimate the 
errors which this will yield. PacLaging-elorage-distribution problems often 
involve hundreds of diJerent produets, each in se%-eral different packages, 
and hence thousands of variablw may be required for an accurate repre- 
sentation. A brief examination of the product-package items often shows, 
however, that they can be grouped by dsssca either according to the facili- 
ties used to produce and/or package Ihcni, their weight or value, frequency 
of dem-md. etc. The effect of such aggregation can tlien bo estimated. . . . 
Items are grouped because of an approdmate similarity of properties and 
hence the tiisperaon of these properties over the group plays much the same 
rote as fluctuations in arrival rate at a server, etc. Since fluctuations tend 
to add like approximately perpendicular vectors, rather than numerically, 
the effects of quite htrgo dispersioas are often considerably Jess than might 
at first glance havo been supposed. For example, if a total of one milbon 
quart cans of different lubricating oils must be esnned per month, it makes 
little difTcreeco what kinds of ods go into the cans provided batch dzes are 
reasonably large. Oning to differences in viacosity, cannbg rates may be 
different for provision for beating may bo installed); however, good bounds 
will be obtained by grouping and udng the average for an optimistic lower 
bound and this arersge plus the staodanl deviation for an approximate upper 
bonnd. 


Changing the Nature of Variables 

The mathematical eharactcrislics of the variable may bo changed 
to eimplify its handling. This may he done in many ways; for ex- 
ample, a continuous variable may be treated as discrete, or con- 
versely; and a variable may be treated as a constant and therefore 
its ffucluntions are ignored. 

For example, in bracketing the lequircd scn'cr capacities in a queueing 
]iroceS5, we may neglect fluctuations in the mput and get a clean lower bound 
by using the unaltered average input lal^ and can get a tentative upper 
bound by using an arrival rata equal to the average plus the etambrd dem- 
tion. If the standard deviation w tiot loo large, and if server capacity is not 
too expenave, this may be good eaovgfi; it will almost always bo good enough 
until other important aspects of Ibe operation have been rimihrly “roughed 

It might be argued that this crude procedure ignores the rare peak fluctua- 
tion which may exceed a storage oi production capacity, etc. Ifowever, in 
most nsses a rigid upper limit on etorage or production capacity is itself a 
model approximation. The nominal capacity of a warehouse cm be exceeded, 
for example, by piling goods on phtfonns or leaving them in freight cars. 
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at an infrequent added cost of extra handling, spoilage, theft and/or 
rage charges, etc.; and production capadty can often be incre^ed at tne 
added cost of overtime, subcontracting, etc. In short, most nominal capaci- 
ties are not rigid but can be increased at a price, and hence we are seldom 
dealmg with a qualitative yea-no question but, rather, with a simpler quan i- 
tative question as to where the balance lies between excess capacity wnicn 
lies idle most of the lime and an occasional but possibly very costly expan- 
sion of nominal capacity. The order of magnitude of these effects can usu- 
ally be estimated to sufficient accuracy to decide whether they need be taken 
into the initial tentative models or can be left for a later stage of refine- 
ment. [Camp (1957-58, p. 11)] 

In some problems involving the amount of equipment required to 
perform a task, it is much ampler to treat the control variable (num- 
ber of pieces of equipment) as a continuous variable rather than as 
one whose value must be an integer. When a noninteger is obtained, 
it is rounded off. In m.any cases it can be shown that this approxi- 
mation introduces only very minor distortion in the measure of per- 
formance. 

The effect of treating the decision variable as continuous in the 
model and of rounding off the solution to the nearest integer can 
usually be determined quite simply. For example, suppose that we 
have the following model: 

r = x+j. W 


in which -Y is the decision variable, and F is independent of A'. We find 
the optimal value of A' as follows: 


dV 

dX 


Y 

X* 


0 


(6) 


X, = Vy. 


( 7 ) 


Therefore, the minimura value of F is 

V, = Vy + = zVy. (8) 

In rounding off we add a positive or negative fraction, a, where 0 < a ^ 
0.5. Then the T’ actually obtained, using the rounded-off value, A'# + Or 
is 


V' = Vy + 0-1- 


VT-l-o’ 


( 9 ) 
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The round-off error ia then 

I” - 1 % = Vr + o + 


= a + 


vr+0 

-Vf 


-2V^ 


VT+<,‘ 

a('/r + a) + r - Vr (Vr + a) 

\/r+ a " v'r+a' 


(10) 


As y approaches infinity, the error approaches zero. As I* approaches 
zero, the error approaches a. 

The replacement of a discrete variable by a continuous one is 
eometimes referred to as ^tdditatioa. Its niivantages are cited by 
Glen Camp (1937-58, p. 7) in the context of problems involving wait- 
ing lines (queues) : 

The "fluid" appra'umstioQ, in which discrete variables iti, tij, . . . repre- 
eentmg the Dumber of items in various queues are repbeed by coatmuous 
variables, is very useful m some coses, suce it coaverts mued derivation and 
finite difiereoee equations into partial derivotive equaiions of dilluaion proc- 
esses, and thereby permits e-tunsive knowledee of the properties of these 
biter in be effectively applied At first gbnee it might appear that this 
approximation would be useful only when the expected queue sizes are large 
conipirtti to 1. In this case, the ‘'Hwd” approximation dose indeed give ex- 
cellent results; however, it can also be u^ to establish upper and tower 
bojends even where expected queue ozes are quite email. 

Situations arise in which what seems to be & discontinuous vari- 
able may be continuous. Consider the ships in a port which require 
loading or unloading. The ship operators must ask for a discrete 
number of gangs of longshoremen because union regulations require 
this type of "order." But the work required is really a continuous 
variable in gangs, which tlte system rounds off to an integer. Hence, 
for example, one may represent the work requirement as a negative 
exponential, even though the number of gangs worked is discrete. 

In some cases a variable can be treated as a constant. TIic value 
selecteil for tlie constant is usually the expected value of the vari- 
able. The outcome may then also be expressed ns on expected value, 
ror exiTOpte, if in the model 
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the expected value of the uncontrolled variable, F, is u«ed, the model 
would generally be written as 

7-Z + |- (II) 

Such a model ignores the penalties associated with values of F F 
For example, in the economic lot-size equation in which average demand 
is used as a constant, the costs of less demand in a specific period (addi- 
tional inventory-carrying cost) and of more demand (shortage costs) are 
ignored. This is equivalent to assuming that the standard deviation of 
demand is equal to zero. The error introduced by this approximation 
depends on how large the standard deviation really is and on the magni- 
tude of the shortage and inventory-carrying costs. 

Modifying the Function 

A model may frequently be simplified by modifying the functional 
form of either the entire model or of expressions within it. Since 
linear functions are the simplest to deal with, particularly when a 
number of functions are involved, as in a set of simultaneous equations, 
nonlinear functions are frequently approximated by linear functions 
so that the matrix manipulation is facilitated. In some cases it is 
necessary to use a linear approximation only in the region of interest. 
In others where the region is large and the function has a great deal 
of curvature, a series of linear approximations is applied to sub- 
regions of the nonlinear function. These approximations are quite 
common where use is made of linear programming, a technique for 
solving a set of linear equations subject to a set of linear constraints 

In a model containing several control variables which is to be 
solved by finding a minimum or maximum outcome by means of the 
calculus, a quadratic form is frequently used because the partial 
derivatives constitute a set of linear equations. In quadratic pro- 
gramming, for example, all cost and payoff subfunctions are assumed 
to be quadratic for this reason. 

Many other functional approximations are commonly used D'®* 
continuous functions are frequently assumed to be continuous, dis- 
crete distributions (e.g., the binominal and Poisson) are approximated 
by continuous ones (e.g., the normal). Limited distributions (e.g-, 
ones with no negative values) are sometimes approximated by un- 
limltcd distributions (e.g., the normal, which allows negative values). 

It should be realized that fitting a function to a set of data is ol- 
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moat always an estimation procedure and hence an approximation. 
TJje researcher, however, e.Ycrcises freedom as to how "approximate” 
his fit is. 

Changing Constraints 

Approximations can he attained fiy the manipulation of constraints. 
These may he added, subtracted, or modified so as to simplify the 
model. 

It is quite a common practice to Ignore constraints in initially solv- 
ing a model to see ahctlier the solution satisfies the constraints even 
when obtained in this way. If it docs, no harm has been done. If it 
docs not, tiic constraints can tm a<tdcd one at a time (in an order 
dictated by niathem.atical convenience) and the model re-solved un- 
til an acceptable solution is obtained. It is sometimes easier to ad- 
just a solution once obtained back into the constrained area than It 
is to find a solution subject to constraints. For example, in deter- 
mining the most economic production quantities for a number of 
products niJide on one nMchioe, the,ee quanllties can be obtained Ig- 
noring the limitc\t time available on the machine. If the sofutions 
oblaincd require more time (ban is available, it is possible to deter- 
mine for each part liow its cost increases as the production quantity 
decreases. ^Vith this information, the originally obtained production 
quantities can be cut back so that the time required is acceptable and 
the cost added by so doing is appro.ximatcly minimized. This, then, 
yields a set of quantities which arc appraximatcly optimal and which 
satisfy tlie time limitations. 

In the biocliemical production process considered earlier, the level 
of production can vary depending on the number of fermenters which 
arc set up. The plant is operated by crows of fixed size. The number 
of crews required to operate the plant is discrete and depends on tJie 
level of prwluction. There is a considerable cost associated with 
changing the number of crews in the plant because of hiring, firing, 
and training. These changes arc complicated by a seniority system 
that requires shifting an average of five workcra every time one ia 
added or suhtnacted from the system. Determination of this cost 
and its inclusion in the model were incredibly complex. In this ease, 
it was much simpler to add a constraint, that the number of crews re- 
main constant. Since it was possible to improve the plant's per- 
formance significantly with tills constraint and because employment 
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stability was desired, the approximation was acceptable to those in 
control of the process. . , 

In general, the omission of constnunts yields an optimistic solu- 
tion, and the addition of constraints a pessimistic one. Consequently, 
by the manipulation of constraints, one can obtain upper and lower 
bounds on the problem solution. 

Problem situations are frequently so complicated that one model 
of the entire situation cannot be feasibly constructed. In such casM 
the problem may be broken into parts so that each part can be modeled 
separately. The interactions between the outputs of the models can 
then be specified and taken into account. We consider first a case 
in which only two models are required, where the output of the first 
becomes the input to the second. For example, consider a problem in- 
volving determination of the number of maintenance shops which the 
Army sliould maintain in order to service inoperable construction 
equipment. 

Fint a model was constructed for determining for any specified 
number of shops, n, the optimal number of men that should bo as- 
signed to each shop, taking into account the fact that, as the work 
force increases, the amount of equipment waiting for repairs decreases. 
The model roinimizes the sum of the shop cost (cost of men operating 
shops) and queue or waiting-line costs. The operation of this model 
is shown schematically and graphically is Figure 4.3. 

This model is used repetitively for a range of assumed numbers of 
shops. For each assumption the minimal total cost is computed. 
These results then become the input to a second model used for 
determining the optimal total number of shops. This second model 
is shown schematically and graphically in Figure 4.4. 

The same procedure of breaking a problem into parts for modeling 
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Fictm 4 4 



is illustrated on a larger wale in the following selection from the 
report of the study from which the previous example was selected.* 

The total expected incrcnientaf ct^t of (he maintenance sj-gtetn (C) can 
be expressed as 

C-P + E-^S + T, (12) 

ffhere P > (lie total expected cost of partt (ordering, purciiase cost, stocking, 
exccfling, etc.). 

B • the total expected cost of the end>ffcni< [major equiproenti in the 
sjitem. 

S •> (ho total expected lAep cost (including direct personnel, indirect 
personnel, tools, faeilitim, ete.). 

7 «» the total expected cost of transporting supported equipment to and 
from the shops. 

Ideally, oise nvuld e.vpand equation hy expressing each of the above 
costs as functions of the controllable and uncontrollable variables. Then by 
mathematical methods, one woidd seek to minimize the total expected incre* 
mental cost by eelecting the proper s-aluca of the controllable variables. How- 
ever, this approach u not mathcmaticnlly feasible. Consequently, an alter- 
native approach was developed. It involve* constructing three "submodels”: 
one lor sitops, one for parts, and one for method of repair [sub-assembly 
^'ereu8 part replacement]. 

In order to break this problem into three parts certain intenictions be- 
tween these parts had to be accounted for. Hist, the number and eize of 
the parts inventory required depend on the number of ahops and hence the 
number of inventory locations. Il was found, however, that for the alterna- 
tive number of shops which are fearible, the effect of this number on parts 
inventory requirements is sufficiently smah so that the optimal solution is 
not significantly altered by toneidering these variable* independently. 

Secondly, the method of repair affects both the parts and shops costs. 
Decisions concerning parts inveatoty and the number and size of shops are 

* Maintenance LogUtiee lor Mejor SJeehmucol Equipment, prepared by the 
Operations Eescarch Group ot Case Institute of Techooiogy for the VS. Army 
Corps of Enfineere, 1900, pp. 0-8. 
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made assuming parts replacement Tlicn the possible improvements to be 
obtained by sub-assembly repiacement arc considered. _ 

A separate submodel js not requjrcd for (be cost of end-items, addi- 
tional end-item requirements (above availability requirements) can be broken 
down into 

(1) equipment sraiting for repairs, 

(2) equipment being repaired, 

(3) equipment in transit, and 

(4) equipment wailing for parts. 

Tlie costs associated with these four types of additional end-items . . . aii" 
pear in tbo shops and parts submodels. 


SEQUENTIAUDECISION MODELS 

In practical problems involving probabilistie variables a model is 
constructed involving assumptions concerning the distributions of 
probabilistic variables relative to cither their form {e.g, normal, 
Poisson, exponential) or the values of the distribution’s parameters. 
At least one additional assumption Is made: that the characteristics 
of the distribution which pertained in the past will also pertain in the 
future. In general, such an assumption becomes less justified os the 
time of applying the model becomes more distant from the time at 
which the model is developed. There are advantages, therefore, to 
making the decision at the last possible moment. 

Put briefly, there is an obvious advantage to making the decision 
as late as possible because at that point of time the maximum amount 
of relevant information is available. There is a very important and 
not so obvious consequence of delaying the decision: it is then pos- 
sible to treat the particular state of affairs at the time of the decision 
and not deal with the "average” characteristics of such situations- 
This frequently allows research economies and better decisions to be 
made. This fact, for example, is Uie basis for the power of sequential 
sampling. In normal sampling procedures, the sample size is deter- 
mined on the bases of previously available information. In sequential 
sampling, the sample size is not fixed in advance, but observations are 
made one at a time or in small blacks and after each observation or 
block a research decision is made as to whether enough information 
is available to make the estimate required or whether the collection 
of data should be continued. Those who developed sequential sam- 
pling realized that the data which are obtained during sampling can 
be used in order to determine more efficiently how large a sample is 
required. As a consequence sequential sampling, where applicable, 
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saves about one-third the obsetrationa required by a fixed sampling 
plan relative to any specified ilegree of precision* 

Sequential-decision models, then, do not “average” over future deci- 
sions situations on tlic basis of past exjwriencc; they deal with each 
situation separately For example, in some inventory situations tlic 
average <lemand and llic average time to replenish stock arc used to 
determine reorder quantities and frequencies. TJiese average quan- 
tities and frequencies are (lien used regardless of what happens in 
any particular period. In sequential-inventory models an advanced 
decision as to when to order (e.g., every week or every montli) is not 
determined, but the reorder point is specified by a certain stock level. 
This level b determined using average replenishment times. But the 
time of ordering becomes a variable in tliis treatment, a variable which 
reflects the characteristics of the penod at hand. Here, then, the 
sequential concept is introduced in a limited way. If the stock level 
at which reordering takes places is also permitted to vary as a func- 
tion of production or purchasing experience during the current period, 
another sequential element is iolrodueed. 

To see tho significance of sequential models in a more dramatic 
context consider a replacement problem involving a large number of 
low-cost items that fail, k number of identical items such as light 
bulbs, vacuum tubes, aud air filters arc Installed at a certain time, 
i B 0. These arc subsequently subjected to usage at an idenUcal rate 
for all. All have tlie same probabili^-density function of length of 
life, /((). Ulien an item fails it is immediately replaced by an identi- 
cal item of age zero. 

Given a population of items such as that described above, it is 
often desired to carry out the replacement process in some optimal 
manner. In military applications it is customary to specify that re- 
placements should be made so as to Qianmizc the mean time to fail- 
ure of the individual items. In industrial applications, where cost 
rather than reliability is Considered, a policy of replacement which 
minimizes the total cost of operation should be adopted. In this illus- 
tration we deal only with the problem of cost minimization. 

Two altem.iljve polieies may be considered: 

(а) Individual replacement of items as soon as they fail. 

(б) Group replacement of all itema plus individual replacement of 
failed items between successive group lepJacemeats. 

•Tlie ma-timum exploitation of avoibble information and its continuous re- 
vision and re-evaluation lie at the root of what is called the Bayesian approach 
to alaUstics. This approach to eslbnatlott theory will be duwissod in Chapter 8 
For further discussion of this type of etaUstics see Chernofl and Moses (1959). 
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The usual procedure of deriwng tlic best replacement policy in- 
volves three steps: 

Step I: Determine the total expected cost (TEC) for the indhud- 
ual-rcplaccnient poli^. 

Step 11: Delcrminc the optimal rcpiacctncnt period and hence the 
optimal TEC for the group-replacement policj’. 

Step III: Choose the policy* with lower TEC. 

Such a policy prorides a complete specification of the group-replace- 
ment times for the future. It assumes that the probability-density 
function /(t) is known, both its form and parameters. There are two 
good reasons to question this assumption. 

(1) If the data on which these assumptions arc based come from the 
manufacturer, the chances are that he obtained them under conditions 
different froia those in which the user will operate. For ecample, be 
may have tested light bulbs by burning them continuously to failure 
under relatively constant environmental conditions. But it is known 
that lamp life depends on frequency of turning the lamps off and on, 
iempenture changes, iamag, and go on; in brief, conditions in the 
laboratory differ from those in the installation involved in the prob- 
lem. 

Furthermore, even if the data were obtained from the installation 
itself, operating conditions may vary significanUy from period to 
period (e.g., winter and summer) and hence the distribution of times 
to failure may vary significantly from period to period. 

(2) The distribution of life spans, even If accurate, is obtained for 
a very large sample, and hence the actual distribution of failures for 
a smaller number of bulbs may vary significantly from it. 

Use of a sequential model involves observing the failures as they 
occur and using this information to decide, at the time of a failure, 
whether to group-replace or continue replacing indiridually. 

We can distingiuah among the following cases: 

SiaMili/ </ /(O 

Catt Form^/IQ FaromrSert e//(l) andTarametera 

1 Known Known .viable in time 

2 Known Unknown Stable in time 

3 Unknown Unknown Stable in tim,» 

4 Unknown Unknown Unstable in time 

Some preliminary research by Rutenberg (1961) on the firet case 
has demonstrated the advantages of sequential models. A technique 
baa been developed wbicb takes advantage of random fluctuations of 
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failures about the mean of » gjMjn distribution; group replacement is 
not carried out at a point in time comspondiDg to the minimum ex- 
pected cost but at a lime at vhich lire actual sequence of failures is 
estimated to yield a minimum cost for that particular sequence. 

Some other problems which can be approached by sequential models 
arc the following; 

(1) A publisher puts out n new book and watches sales week after 
week. On the basis of the sales infornuitton he wants to make tlic 
following decisions: should a new edition bo printed, and, if so, how 
large? 

(2) A theater puLs on a new show and counts the tickets sold each 
nigljt. Should the show be conlinueil or dropped? 

(3) An enemy missile is delected on tlie radar screen. As more 
radar obseni'ations are obtained, its trajectory can be determined with 
better accuracy; on the other hand, precious time ia lost. IMien 
should the countermeasures be activated? 

(4) In certain ser%'ieing operations (eg., operation of a fleet of 
delivery trucks) ndditioos to facilities can be made on short notice 
(eg., by renting trucks and hiring drivers by the day). Here, too, 
observations on accumulation of units waiting for service can be made 
until a change in requirements is indicated. Hcncc, there is a pos- 
aibllity of using sequential-decision procedures for a large class of 
qftevinq problems. 

A number of characteristics are common to the above examples: 

(а) The information available at the outset is general in nature [a 
general demand function of a probabilistic nature in examples (1) 
and (2), the laws of motion and error function in example (3), and the 
distribution of arrival and service times in example (4)], but the 
parameters are estimated, not known. As more information is col- 
lected, a more precise cstim.atc of the parameters may be obtained. 

(б) A cost is attached to errors of estimation. 

(c) A cost is attached to deferring the decision in order to collect 
additional information. 

(d) It is desired to obtain a sequential-decision rule which will opti- 
mize an objective function. 

The methodological implications of such a decision rule are as fol- 
Jowi; 

(1) Control of the solution is inherent in the decision process itself 
instead of being dependent on an external-control procedure such as 
is discussed In Chapter 13. 
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(2) Situations in which very little information U 
outset can nevertheless bo approached in a systematic as . 

may undergo continuous improvement as time goes on and more - 
information (such as the distribution of parameters) 
able. Thus, in effect, a learning process is introduced in the - 
a Ecquenlial model. 


THEORIES, LAWS, AND DEOSION MODELS 

Theories and laws in pure science are frequently expressed in the 
form of mathematical models. Such theories and laws may play 
important role in decision models because the phenomena invove 
in the problem situation may behave in accordance with 
theories and laws. For example, a decision model which is conslroctca 
for use in selecting between allcrnative designs of equipment wi 6 
erally contain physical laws which relate the performances o 
various types of equipment to their characteristics. 

The discussion of various types of models and approximation 
contained in this chapter is also applicable to the constructiw o 
quantitative theories and laws. Here too mathematical managea 
and accuracy must be brought into balance. For example, 
Ptolemaic (epicyclic) geocentric theory was replaced by the * 

can (circular) heliocentric theory largely because of the greater ma e 
raatical manageability of the latter , 

The pressures of time and the limited mathematical abilities 
those who use the research output arc not likely to be major factors m 
determining the form of pure laws and theories. But even in pure 
science research resources (men, money, material, machines, an 
time) are likely to be limited, and short cuts may be necessary. 

It is important to realize, however, that the possibility of future 
improvements of decision models, and of pure laws and theories, e 
pends critically on the explicit formulation of the approximations 
which have been made and of the assumptions and evidence whic 
justify or explain their use. It should become increasingly feasi c 
to evaluate approximation procedures and to develop general prm 
ciples for model construction to guide researchers into "best” approxi- 
mations in any given situation. 

For detailed discussion of laws and theories in science see Camp- 
bell (1952 and 1957) and Churcbman (1961). 
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CONCLUSION 

It should be apparent that there is considersble flexibility in model 
construction. It is unlikely that different researcJiers confronled by 
the same problem will emei^ with identical models, although their 
models should be similar in most cases. As yet few principles have 
been developed to assist the researcher in this phase of his work, but 
a means for clcvelopine such principles is becoming increasingly 
available; simulation of a wide variety of even complex phenomena 
on electronic computers. It is possible to use a simulated phenomenon 
as "reality” and to try different moilcling techniques on it. Since, in 
such cases, "reality” is known, it is possible to evaluate aUemative 
models and ultimately ta extract principles of model construction from 
such trials, if they are conducted in a controlled way. The simulator 
can also serve as an effective pedagogical instrument In teaehiag stu* 
dents of science and engineering how to construct models. 

Two aspects of a decision situation, in which research is being per* 
formed for the decision maker by others, are not usually, if ever, in- 
corporated in the model even approximately: (1) the probability nsso* 
eiated with each possible course of action; that is, llie probability that 
the decision maker will follow that course if the research so indicates; 
and (2) the likely amount of distortion the course of action will be 
subjected to in application if accepted. The reason for the omission 
of these aspects of the decision maker’s behavior is obvious; llie 
inability o£ the researcher to quantify tliem. To be sure, the effective 
researcher takes these factors into account in deciding wlial ajiproxi- 
mations to use and in designing Uie implementation of a solution de- 
rived from the model; but he docs so qualitatively and often un-self 
consciously. The science of p^'chology will have to advance a great 
deal more before these factors can be handled quantitativ-ely and 
consciously. Xn the meantime they can at Ic-ist be handled con- 
sciously, and ways of doing so arc discussed In Chapter 14. 

Construction of a model involves continuous testing of its adcqu.icy. 
To see what is involved in the testing procedure we consider the ways 
in wliich the model can be in error. 

(1) The model may contain vnriablea which arc not relevant; that 
is, hai'o DO effect on the outcome. Thdr inclusion in ilic model, then, 
makes the predicted outcome depend on factors on which it has no 
dependence in reality. 
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(2) The model may Eot include variables which are relevant; that 
is, ones that do aQect the outcome. 

(3) The function, /, which relates the controllable and uncontrol- 
lable variables to the outcome may be incorrect. 

(4) The numerical values assigned to the variables may be mac- 
curate. 

If these deficiencies are a result of ignorance rather than of deliber- 
ate approximation, they should be uncovered and corrected. In the 
following six chapters we shall consider each of these types of error 
in detail and procedures for mmimizmg them. First we will consider 
the evaluation of variables. This discussion will involve the subjects 
of defining, measurement, sampling, and e$timation. Next we will 
consider the relevance of variables, and this will include a discussion 
of correlation analysis and causal analysis {experimentation), and 
the testing oj hypotheses. 
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would be observed if certain operations arc performed under specified 
conditions on specified objects. , 

The conceptual definition of '•father," for example, would be maie 
parent." In this definition Uic concept "father" is equated to the con- 
cept "male parent” and hence may be written as 


Father = male parent. 

In an operational (icrmilion ot "Intlier," lionicvcr, the concept would 
be eciuatcd to some function of one or more obscr\'ations made un er 
certain specified conditions. i j r • 

By showing the relationship hetween concepts in conceptual e m 
tions the relevance of various types of knowlctlge to the study o c 
concept being defined is demonstrated. Such definitions allow iis o 
develop a conceptual system through which the various types o in 
quiry in science arc related. In a later section of this chapter u® 
aspect of defining will be illustrated when we begin with concepts o 
geometry, kinematics, and mechanics and derive ft wide range of ot cr 
types of concept from them. _ _ 

In a sense, then, conceptual definitions tell the scientist what to 
think about in relation to n concept, and operational definitions tc 
him what to do about answering questions involving the concept- 


OPERATIONAL DEFINING 

According to Bridgman (1927, p. 5) "in general, we mean by any 
concept nothing more than a set of operation; the concept is synony- 
mous with the correspondwg set of operations.” S. S. Stevens 
p. 323 ), operationalism’s principal spokesman in the behavioral sci- 
ences, echoed, "Operationisro [he contracted the term) consists simp y 
in referring any concept for its definiUon to the concrete operations by 
which knowledge of the thing in question is had." The first task o 
the operationalist is to define an operation and, as Stevens observe 
( 1935 , p. 323 ), “If we are to be consistent, we must define it opera- 
tionally. An operation is the performance which we execute in order 
to make known a concept.” 

Further, according to Brid^nan (1927, p. 6) , "we must demand that 
the set of operations equivalent to any concept be a unique set, for 
otherwise there are possibiiities of ambiguity in practical applications 
wliich we cannot admit.” It is pointed out that different sets o 
operations give different concepts even though the results may be the 
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same. When such is the case, the concepts are considered to be 
“practically’’ the same, thougit not “actually’’ so, whatever this dis- 
tinction means. 

Strictly speaking, length when meosuied fay Irght beams should be called 
by another name, since the oprratiom arc dilTercnt [from mcasiiring by a 
measuring roiij. The practicaJ justification for the same naiuo is tJiat wjlhin 
our present experkncntal liraiu a numencai difference between the results of 
tlie two sets of operations has not been detected. (1027, p 10) 

Bridgman provided no criterion for determining wlien two sets of 
operations are the same or different. Consequently, it is difficult to 
see how two sets of operations can ever be exactly the same in everj' 
respect. Tliey must at least differ in either time or the operator — 
and if they need not be exactly the same, then by how much and in 
what respect may they differ? 

Concern with such questions led Stevens (1035, p. 325) to note: 

If the eye could distinguisli between lengths of inlinitcsinia! sire, it would 
be impossible to find two objects between whose lengths the eye could distin- 
guish so difference. However, the capacity for discrimination is biimas ob- 
servers is limited, and aUo variable This lunitatios has as a direct conse- 
qucnce the fact that all operations arc at Iwst only approximations. A 
penumbra of uncertainty surrounds even Uie most prcci.<£ and obvious 
operations. 

Hero Stovens admits that discrimination is subject to error, and 
that DO precise specification of on o|)eration can over be perfectly 
satisfied. Yet bis doctrine asserts Ibnt we must restrict ourselves 
to operations which cun be pcrformcti: "Science ia concerned with the 
formulation and dctennioalioo of concepts which arc based upon 
operations which can be carried out" (ld35, p. 324 fn). It is there- 
fore implicitly admittnl since no precisely definod operation can 
be o.arried out without error, such opemlions are ininffinarj/. But, 
according to Stevens, "to sflmit Ihc posribility of defining cerfain 
concepts by ‘imaginary’ operations . . . detracts from the rigor of 
ofierationisoi «s a procedtrre for detennining empirical definitions'’ 
(1035, p. 324 fn). 

To determine whether or not two operations are the same raquircs 
that we make explicit the sense in which we want samenesa to be taken. 
Tin’s requires an explicit st.'itcment of the purpose for estabJishltjg 
difference or similarity belwecti two or more operations. 

These comments on the ambiguity and inconsistency of opcralionul- 
ism as it relates to defining bavo more than academic significance. 
It can safely Iw said tliat there arc few problems in contemporary «}- 
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ence as critical as that of preparing instructions which will be carmd 
out in the “same” way by different observers. This point has bee 
illustrated over and over again in the so-called “exact” sciences. 

For example, in World War 11, the “same” type of a specific article 
was made in a number of different places. The government, wliicli 
“consumed” these articles, attempted to set up vety precise mstnic- 
tions for measuring their “hardness.” It set up “round-robin tes s 
to determine whether different laboratories obtained the same results 
on the same items by following the “carefully worded" set of instwc- 
tions. In this particular case, as in many others, the results obtained 
by different laboratories were so varied as to invalidate the ^ ® ® 
process of testing. One "simple” operation, for example, invol«d 
counting the grains in a well-marked area on a brass strip. ® 
laboratories differed significantly in their counts. Apparently t ere 
is something in the individual laboratory’s training of its ob8er\er3 
which is “added to" the instructions. _ . 

One might argue that, in the case cited, the instructions were no 
precise or complete enough. But, If we want to avoid the tautology; 
"Iilake them precise and complete enough to give reliable results, we 
must turn to some other theory of definiog. Stevens tried to avoi 
such an alternative by seeking an ultimately simple operation tha 
could be performed identically by all. \Mic*n operations are not under- 
stood by someone, he argued (193a, p. 327) : 

We should then describe the operations in Icnns of others which 
simpler to him by reason of his own experience. Three operation* may sga 
be defined in terms of still simpler operations until we come at last to oper- 
ations winch seem to be at present, though not necessarily alwa>s, fim y 
mental and unanalysable. This process might be called the ojieratuy^ 
gress. It is not infini te. In practice, the operational regress need be pu ^ 
sued only until agreement is reached. . . . Description of a concept 
at a certain level, be made in terms of constructs with little obvious relaUo 
to what we see or hear, but, when the description of the constructs “C®" 
selves must be gi«n, inevitably appeal is made to objects or events to whic 
we can point. 

The inadequacy of pointing as an ultimately simple operation, boW' 
ever, is dramatically illustrated in the following anecdote of J- “• 
Weeks,* related by Ogden and Richards (1947): 

I remember on one occasion wanting the word for Table. There 
or six boys standmg round, and, tapinng the table with my forefinger, I sskm, 
‘TVhat is this?" One boy said it was a dodela, another it was an elanda, 

* J. Q. Weeks, Among Congo CamSmU, p. St. 
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a (bird stated that it was 6oAa(i, a fourth that it was elamba, and the fifth 
said it was mera. These various nords we wrote in our notebook, and con- 
gratulated oursrlvpft tiiat we were workup among a people who possessed a 
language that had five words for one article. 

At a later stage Mr. Weeks ihscovcred: 

One lad had thought ne wautcil the word for tapping; another under- 
stood we were cocking the won! for the maten.tl of which the table was 
made; another had an idea tl«t «e required the word for hardness; another 
thought that «c wished for a name for that which covered the tabic; and 
the last, not being nhlo, perb-apa, to ilunk of anything else, gave ua the word 
mei8, table— the very word we were seeking. [Ogilcn and Richards (1047), 
pp. 77-78] 

A case has also been reported in which Uie informants, instead of 
giving disparate names to the “same" object, gave the same name to 
every object at which the anthropologist pointed. It turned out that 
they were giving the word for “finger.'’ 

In order to avoid aueh ambiguity in pointing we would at least have 
to confront the observer with two things different in every respect 
save Uio one we were pointing to and then make clear that we wanted 
hitn to abscr\'e the eommoo property. This is not only not simple; 
it is also impossible. If operational deffning must depend on the 
eaistence of a set of irreducible and completely unambiguous opera* 
tions, ii is not possible. 

Bridgman was aware that, if operational defining is a process of re- 
duction, wbat w'c ultimately reduce (o is not a set of operations but a 
Set of assumptions. For example, in discussing the measurement of 
distance between too widely ecparaled points he observes (1927, pp. 
14-15); 

IVe assvme tb.it a beam of light trorrb in a straight hoe. FurtJienuDre, 
Mc assume in extending our eyetem of tiiangulation over the surface of the 
earth that the geometry of li^t beams is Euclidean. 

Therefore, scientific definitions, like laws and facts, are not isolated 
islands floating in the sco of science, but are bits of grotmd firmly 
anchored to the land mass of scientific theory, laws, and facts, and 
hence are no leas subject to change than any of these. 

In ‘'reducing'' a concept to a set of assumptions and a set of (not 
irreducible) operations, the defiaer has « great deal of choice. What 
considerations should dictate these choices? Bridgman suggested an 
answer the implications of which he never fully explored. He observes 
that the assumptions "to which we made reduction and which we ac- 
cept as ultimate . . . depend to a certain extent on the purpose in 
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vieu', and also on the range of previous physical experience {19-7i 
p. 39, italics ours) . 

Defining, then, is not an impersonal mechanical operation; it is a 
operation imbued with all the richness of both the researchers pur- 
poses and experience and, as we shall see, his cultures purposes an 
experience as well. 

Bridgman and Stevens failed to provide a workable theory of opera- 
tional defining, but they made a significant contribution to the d^ 
velopment of such a theory through their insistence that concepts wed 
in science should be operationally defined. They were reacting agains 
the exclusive use of conceptual definitions which do not directly re a 
concepts to experience or experiment. They tended to neglect the im 
portant function which conceptual definition can perform, but it js 
not hard to understand why. In the newer branches of science, in 
particular, it has become increasingly common to define one conMp 
in terms of others which, if anything, are less well understood t an 
liie one being defined and whose operational significance is even roore 
obscure. This is illustrated to the following definition of a personality 
‘'trait”: 

... a generalized and focalized ncuropsycbic system {peculiar to 
viclual), with the capacity to tender many stimuli functionally equivalent, a 
to initiate and guide consistent (equivalent) forms of adaptive and expresse 
behavior [AUport (1937, p. 295)). 


Bridgman and Stes'cns object to such a definition because it does no 
tell the scientist how to determine whether or not he is obser\'ing a 
trait. Although the details of their own proposals for defining pro- 
cedures have been beset by difficulties of their own, the validity of the 
objection to traditional defining which tlieir theory tries to overcome 
has been sustiined. 


One of our objectives in this chapter, then, is to develop an opera- 
tional thcorj’ of defining which provides a more satisfactory basis 
for determining what the content of conceptual and operational defini- 


tions in science should be. 


THE CONTENT OF DEFINITIONS 

A definition, of course, is made up of words, and these words convey 
meaning. In determining what meaning the definition of a concept 
should convey two considerations arc involved: (1) the purposes of 
the researcher for which the concept is being defined (t.e., bow and for 
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what he intends to use tiic concept), ami (2) the way the concept has 
commonly been usetl both in and oat of science. 


The Purposes of the Defmer 

The relevance of the purposes of inquiry to scientific defining had 
been recognlred Jong before Bridgman observed it. For example, 
F. C. S. Schiller {1912, p. 71) wrote: 

■^e essence . . . which every definition ines to state is simply the point 
which it js for the time being important to elucidate. It foUons that the 
escncea and definitions of things ore neees&anly plural, variable, and "rel.a- 
!ive” and never "absolute". . . . A ringle. unnibtakable and absolute defini- 
tion of a thing, true without refcteiiec lo aiq' context, would liaw to be one 
that would serve for any purpose for winch it w ronveniemt or possible to ii.‘=e 
the terra. Such a definition is barely coueeivable, hut quite incredible, anil 
BKuredly not extant. 

Schiller's observations arc illustrated by an experience of tlie author, 
who was once involved in determiniog the accuracy of results ob- 
taioetf Jr a survey dcaiinecf to dctcsmine (he Dumber of rooms Jn 
dwelling units. The survey bad been conducted without an explicit 
definition of ‘Voom.” He met with (he designers of (lie surx’ey and 
asked what definition they had used implicitly. They were impatient 
with the question, observing, "Everyone knows what a room is.” The 
author persisted, and one of tlioso present ofTered: ‘‘A room Is a space 
enclosed by four walls, a floor, and a ceiling." The conversation then 
proceeded mucJi as foilows: 

The author asked, "Can’t « room be ttunjular?" 

"Sure. It can have three or four w^s " 

"What about & circular room?" 

“Well, it can ha>e one or more walls." 

"What about a paper c.artonl” 

"A room has to be large cnougfi for human <icc«p.sney." 

“What about a closet?” 

Tt must be used for normal living purposes." 

"What are 'normal Lving purposes’?" 

"I<iofc, we don’t have lo go through this nonsense; otir results are good 
enough for our purposes." 

"What were your purposre?" 

'To get an index of living eondit'ions by finding tlie number of persona per 
room in direVing units." 

"Doesn't the sisc of the room matter?” 

"Yes, we probably should have used ‘eqnare feel' of floor space, but that 
would have been too hard lo get.” 

"Doesn’t the height of tlie room matter?" 

"I guess so, Ide-ally, we should have used volume." 
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"Would a room with 10 sqiure feet of floor area and 60 feet higb be the 
eame as one with 60 square feet of floor area and 10 feet high? 

"Look, the inde-r is good enough for the people who use it. 

"What do they use it for?" 

"I’m not sure, but we've had no comiflaints." 

This conversation makes clear that without an explicit statement 
of the purposes of tiie inquiry Uiere are no criteria of the adequacy o 
defining. 

The accuracy of an estimate of the average number of rooms per 
dwelling unit, for example, depends at least in part on the precision 
of the definition. On the otiicr hand the degree of accuracy required 
can be used to determine how precise tlie definition has to be. 1 
want an estimate of the average number of rooms per house in a ci y 
that is within 10 per cent of the true value, we require a less 
definition than if we want an estimate within 0.1 per cent. In the rs 
case we might not worry, for example, about enclosed porches, u 
these might be critical in the second case. 

The role of purpose m this sense is as relevant in pure **;‘*^® 
as in applied science. In the former the scientific purposes dicta e 
the degree of accuracy required. , 

This is not to say that definitions need only serve the rescarene 
or decision maker’s purposes; they must 8er\’e the comnuinica lo 
obiectives of science and society as well. For this reason definitio 
must also be related to past and present usage of the concept involve 

Past and Present Usage of Concepts 

The meaning of a concept should, in addition to serving the research 
purposes, take cognizance of historic and current usage. Otherwise 
the objective of communicating results, which is present in every rc 
search effort, will not be effectively realized. 

The meanings of concepts evolve and arc as dynamic as any o 'C 
aspect of science. Past definitions rc'.’eal intended meanings. T ’cy 
may not, however, have successively added precision to the meaning 
the concept. One of the purposes of this discussion is to establii ^ o 
procedure which, if followed, would tend to increase the rate at w ic 
precision in defining is attained. , 

Historical analysis of the use of a concept can often reveal a tren 
in the evolution of the concept or a consistent theme of meaning whic ' 
persists through numerous variations. This is very well illustrated y 
E. A. Singer's (1929) penetrating analysis of the meaning of con 
sciousneis. Unfortunately the analysis is too lengthy to reproduce 



DETINIKO 149 


here in full, but perhaps the following condensation will give some of 
iU flavor. 

Singer obsen-es (p. 508} that the history of the concept can be 
divided into two parts: 


... a classic expansion from a most special to a most general use; and a 
modem . . . contraction from a most general to a most special u«c. ... It 
shall be shown (I) that while the story of the conscious carries an idea from 
a narrowly restricted beginning to a narrowly reslneted end, yet never was 
the idea further from its first stale than in its List; and (2} that t^hough 
the conscious thus grows and dimimshM in extension, there yet runs through 
its history a continuous thread of inleriiion. In this unchanging implication 
of the comcious lies its real value to humanity. . . . The task it was bom 
to perform . . . la the task of distmguuhiitff the slate o] mind hwwtng an- 
other state of mind. 


Singer notes and documents a succession of three classical uses of 
the concept: 

(1) The primitive meaning, Ittowing somtlhing with another: 
. . in its primitive meaning the term requires at least two knowers 

if one ia to be called conici(M . ■ •” (p- 569). 

(2) A more gencrnl meaning, obtained by e.\tcnsicn of tlic meaning 
to include fcnoitnnff something in one’s self. 

. . . "knowing with” is made to include not only the ease of knomng with 
another self, but also lie case of knowing with ones other self . . (p. 569). 
. . The aenwotic transition from the conscious sharing of anolhers mind to 
the conscious elianng of one’s own must haie eonie about without eenable 
olt or S of invention. ...» with like continuity the gJP "fmory /s 
supposed to span in rendering a pr«ent co««om to its piut fe 
to^Sninish by insensible degrees, there is no 
lose meaning till the limiting case Mn be mehed 
for the mathematician to require llmt 

of a convergent series ehaU aUo hold good for the limit, (p. 570) 

This lindting case providea (3), the most general possible meaning 
of conscious. Here the term includes not only the cas^ of knowing 
with another and witli oneself one’s own p-nst, but also knowing ones 

'’"TuSSriolh^modem period. Singer observes (pp. 571-572); 

it: 

very order of their of ii^m^ate sclf-knowl- 

!or conscious is retained save that laniu^s- 

edge to which the Latin consoous semu nevw to h-ave attained. 
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Singer documents tliis modem contraction of moaning of the con- 
scioui and concludes his account with the following quotation from 
Reid (Essays on the Intellectml Pmters of ^[an, 1, 1, 1785, pp. 17 f.) : 

Comciovsnesa is a word used by philosophers, to signify that iininediate 
knowledge which we have of all our present thoughts and purposes, and, in 
general, of aU the present operations of our mind. Whence we may observe, 
that consciousness is only of things present. To apply consciousness to 
things past, which sometimes is done in popular discourse, is to confouM 
consciousness with memory; and all such confusion of words ought to t* 
avoided m philosophical ihscoujse. 

Having concluded liig analysis. Singer ohserves (p, 574) : 

The story of the conscious carries its hero from a narrowly restricted be- 
ginning to a narrowly restricted end, the first and last state of that word 
bemg of all its states the most opposed in character, while through this 
varied story runs a continuous thread of impiicatioo: being conscious is the 
state of mmd knowing another stale of mind. 

He then uses tliis analysis to construct hw own definition of conacienu- 
Tieat, which retains thU historic thread but converts the concept into 
one that has operational and observational significance. 

This discussion of the content of definitions may be summarized 
by the following instructions: 

(1) Examine as many definitions of the concept, past and present, 
as possible. Keep in mind the chronology of the definitions e.xamined. 

(2) Try to identify the core of meaning toward which the defini- 
tions seem to be evolving. 

(3) Formulate a tentative definition based on this core. 

(4) Examine usage of the concept in the context of the problem or 
question to which tlie research is directed and determine if the mean- 
ing you have formulated will serve the decision maker’s or research 
objectives. If not, make necrasary revisions. 

(5) Subnut the definiUon to as wide a critical appraisal as possible 
and make any justifiable revirions suggested by the criticism. 

We have a final observation relative to the purposes of defining: 
if two different (physical) sets of operations yield results which are 
equivalent from the point of view of the purpMes of the determina- 
tion, then these sets of operations are functionally equivalent; that is. 
practically it would make no difference which set is used. It is P®-" 
eiblc, therefore, to have two (or more) different but equivalent opera- 
tional definitions of a concept. No one has argued against the possi- 
bility of there being two or more different but equivalent conceptual 
definitions of a concept. 
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FORM OF OPERATIONAL DEHNITIONS 


Tlic notion of the operHlionalisU tliAt sciciitifie (icfinilions should 
specify operations is one (hat wo wiH try to (wscn'c without incur- 
ring the difGculties in their thcoiy. Operational definitions should he 
directive; they should loll the researcher Jjoiv to investigate tljat which 
is conceptualized. It is not enough to provide definitions which 
"clarify” a concept; not only should conceptual tiefinitions employ 
more underelandablc concepts tlian the one being defined, but also the 
concepts used should themselves be capable of receiving operational 
definition. 

The traditional approach to clarification of meaning has consisted 
in "reducing'' a concept to be defined to a set of concepts whose mean- 
ing is immediately known without ambi^Uy and uniformly over the 
population. The search for aucji primitive concepts, like the search 
for primitive operations, has been fruitless. [For a detailed discus- 
sion SCO Churchman and Ackoff (1&50, Chapter XIII), J Concepts 
are required to specify operations and operations to specify con- 
cepts; it is only through the interaction of concepts and operations 
that meaning is clarified. Suppose that we follow up our earlier 
discussion of tliis point, and contrast the following definitions: 

(a) An individual is said to possess, or to be characterized by, a certain 
personality trait when bo exhibits a generalised end consistent form, mode, 
or type of rtaclirity (behavior), and differs (deviates) suffleiently from other 
members of his social environment, both in the frequency and intensity of 
beharior, for his atypicality to he noticed by relatively normal and impartial 
observers, themselves members of the same environment . . , [Vernon (1933, 
p. M2)]. 

lb) Tbs latcmHionaJ Prototype Metre is a bar of platinum iridium alloy 
(90% platinum, 10% iridium) a special winged X-form section devised 
by G. Tresca to give maximum rigidity in relation to the weight of met.al 
used; the neutral plane of the section i? exposed throughout the length of (he 
bar, and the metre is defined as the distsnee betivccn two transverse grad- 
lufions on the neutral plane, near the oids of the bar, when the latter is at 
the temperature of 0° C. . . . [Darwin, Sears, et of. (IMS, p. 152)] 

It is neces-iry, of course, to matntaiii a close control and make accurate 
measurements of the temperature of the bars during comparison. For this 
reason the comparator is arranged » that the bars can bo immersed, during 
measurement, in the inner compartninit of a double water-bath, provided 
with stirring devices to maintain a uniform temperature. . . . Since differ- 
ent bare will have different coeffidents of tlsermal expansion, it is necessary 
to determine these and make olhnnuKes for them in computing the results 
of coaparisons actually made at other Iciaperatures. (p. JM) 
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Althougli it is true tiiat tlic first definition (n) is vague in its 
reference to operations, and (6) is precise, we note that (6) docs not 
tell us how to detcnninc in any practical situation whether or not an 
object is one metre long. Tliis definition tells us how we ought to 
make such a determination in an ideal situation. Such defining speci- 
fies the best conccicohlc (not necessarily obtainable) conditions under 
which, and procedures by whicli, values of tlic variable can be ob- 
tained. “Best conceivable” is, of course, ft relative concept; it means 
the best that u e enn conceive, not the best that will ever be conceived. 

At first glance, such a step might appear to be very impractical. 
Why botlier to specify operations which we may not be able to carrj' 
out? The answer lies in the fact that specification of ideal (or op- 
timal) obsen’ational condibons and procedures is finite important if 
we want to know how good are the results we eventually obtain. 
Furtber, and more important, the ideal conditions and procedures act 
as a tlandard by means of which we can compare obscr\’ations made 
under different conditions using different operations by making adju$l- 
meals in the results so that they represent wliat would have been ob- 
tained under the standard conditions. The use of idealized operational 
iefinitiona for the adjustment of actual data is not new to scienee. 
Consider a familiar physical example: the detcrminalion of the ac* 
celeration of a freely falling body. The idcallrcd operation for meas- 
uring such an acceleration requires (among other things) a perfect 
vacuum in which the body can fall with complete freedom. Actuallyi 
the physicist can never create a perfect vacuum, but he can conduct 
his e.xperimeDt in such a way as lo estimate how a body would fall if 
it were in a perfect vacuum. lie determines how acceleration is af- 
fected by variations in the density of the atmosphere. He uses 
mathematical functions to relate changes in the atmosphere to changes 
in acceleration. Then, by extrapolalioa, he determines what would 
occur in a complete vacuum, and therd^* infers the acceleration of an 
ideally freely falling body. 

An operational definition of a concept, then, should state crplictfly 
the condilioni under which and the operations by which questions 
conccmtnq the concept mrolted ought to be (ideolly) answered 
This notion of operational defining does not require fundamental or 
irreducible operations or concepts. It requires the specification of a 
standard procedure under standard conditions, deviations from which 
can be measured. The effect of these deviations on the results ob- 
tained should be determinable. Consequently, effective scientific de- 
fining requires capabilities for measurement and adjustment of data. 
Progress in defining, then, is closely related to progress in measure- 
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ment and in Uie development of tlie theories and laws on the basis of 
which adjustments of data can be made. The latter constitute the 
definitional assumptions to uliicii Bridgman refers. 

There arc many rases j« scienn* wlicre we ran rJoeJop the stand- 
ards necessary for good operational derining but where we may not 
yet have either the nei essary teelmK|iies of measurement of deviations 
from tfiese standards or the llieory and laws necessary for adjusting 
for these deviations even where thej’ can be measured. The definition 
of “value” developed in Chapter 3 is a ease in point. But by con- 
structing such definitions we point up deficiencies in available meas- 
urement techniques, laws, and theories, and by so doing we acceler- 
ate their development Until they are developed, we may have to 
use very rough approximations or even guesses, in order to adjust 
data, but if these approximations are made explicit they may attract 
the attention of other researchers and hence increase the chances of 
research effort being directed to them. 

Types of Variables and Constants 

The variables and constants which appear in a mode], tlieory, or 
law are essentially of two types: (1) a numher of objects or events, 
a count or enumeration; and |2) the a»»o«nt of a property which an 
object or event possc.sses. For example, tiie arrival rate of customers 
at a service point is the numlier of customers arriving per unit time; 
the duration of a process is the amount of time (the number of units 
of time) it takes. Ilcncc, in defining the variables and constants 
which appear in a model we define objects, events, and properties of 
these. 

As we shall see, objects and events are defined in terms of their 
essential properties, and hence all scientific definitions should “reduce" 
to operational definitions (in the sense we have developed) of prop- 
erties. Before considering the format properties that such definitions 
should have, however, some obscrs'ations about definitions of objects 
and events are necessarj'. 

Objects 

In defining objects for tlie purpose of counting them it is necessary 
to define tlie class of objects which are to be countetl. This means 
specifying The properties which are sufficient for determining whether 
or not any particular object is a member of the efass of interest. This 
may be done in cither of two ways: (1) by identifying the objects 
to be counted, a denotative definition, or (2) by individuating the 
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class of objecls, a connofativc definition. Identification is normally 
nccomplialicd either by fpccifying the npacc*liinc cooniinates of an 
object or by naming it, where apjiIicabU*. Tims we tan speak of 
counting all the objects on u particular pbcH at n particular tune or 
of counting all the people namrtl Smith. Tlie ability to identify all 
the relevant objects in an inquiry, however, presupiwscs ii critenon 
of relevance, and this in turn is ha*cd att n sficcififation of ‘'essential 
properties. Ultimately, then, a definition of a class of objects in 
terms of its essential properties is required. Once w'c have such a 
definition we can resort to identifying the members of the class. 

The "cssentiar* properties of an object arc tliosc properties which 
are individually necessary and collectively sufficient for inclusion m 
the class to be counted. For example, “four legged” is not a neces- 
sary property of a chair, since some chain may have three or other 
numbers of legs. ‘'Having a Lack,” however, is ncccssarj', for if we 
remove the back of a chair it tceomes something else, a stool. This 
property, however, is not sufficient for being a chair because a sofa 
also has a back. Thero/ore, another necessary property of a chair i* 
that It is intended for use by only one person at a time. 

The scientist cannot afford the looseness that rharactcrites dictionary 
definitions. For example, one dictionary* defines a chair as “a scat, 
usually movable, for one person. Ti usually has four leg-s and a back, 
and may have arms.” Tlic scientific definer must decide whether or 
not Uic four legs, arms, and back arc necessary. This decision must 
be made in light of the research objectives and common u«agc. If 
the researcher’s purpose is to determine tlic number of people who can 
now be seated comfortably in a specified room, his definition might be 
quite different from that used in a situation where comfort is not an 
objective and a future date Is Involved. 

The definition of a class of objects, then, should consist of a speci- 
fication of properties c-ich of which is necessary end all of which are 
sufficient for diflerentiating the class of inlercst from all other classes 
The selection of these properties should be dictated by the research 
objectives, and the name by which the class is identified should be 
chosen with past and present tcnninology in mind. 

Events 

An event is something which happens to one or more objects That 
which happens can always be described in terras of a change in prop- 
erties. For example, the meeting of two people can be defined m 
terms of changes in their locaUon or their awareness of each other. 
An object can be said to be btimuitr when certain of its pliysical 
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properties change in a specified way. A drill can be said to be duiffaff 
when its rate of culling decreases, and so on. In defining an event, 
then, it is necessary to specify the object(s) involved, their relevant 
properties, and the changes in these properties to bo observed. 

Types of properties 

There are a verj- large number of ways in which properties can be 
classified, most of which have no methodological significance. One, 
however, does: the classes based on the distinction between ttnictural 
and functional properties. In loose common terms a Btructunal prop- 
erty is one which refers either to the matter (or material) of which 
a thing is composed or to its form, or botli. For example, if a certain 
tjTie of ball is defined ns a “round rubber" object, both “round” and 
“rubber” are structural properties. Functional properties, on the 
otijcr hand, refer to how a thing or event came into being, or what 
it does or can be used for. For example, another type of ball is de- 
fined as a “globular missile.” “Globular” is a structural concept, but 
“missile” Is functional, siocc it refers to a use to which the ball can 
be put. 

Although one might be inclined to think ol structural concepts as 
the domain ol tlic physical sciences and functional concepts as the 
domain of tlie life and behavioral sciences, this pairing breaks down 
rapidly on reflection. For example, a catalytt is usually defined as a 
substance which accelerates a chemical reaction and which may be 
recovered practically unchanged at the end of the reaction. The es* 
scntial characteristic of a catalyst, then, is what it does rather than 
its form or substance. Most physical instruments (e.g , scales, bal- 
ances, tliermometers, barometers, ohmmeters) are defined in terras of 
what they do or arc used for rather than in terms of their structures. 

As a consequence of tlie growing importance in tlic physical sciences 
of such concepts as coraraunie.ation, control, and systems, physicists 
have become increasingly self-conscious relative to Uieir use of func- 
tional concepts. This new awareness was evidenced in a classical 
article by Roscnblueth, ■\Yieacr, and Bigelow (19-13) and in the can- 
temporaiy’ philosophy of science dmTlopcd by E. A. Singer, Jr, (1959), 
In the Ilosenblueth et of. article it is made clear that functional con- 
cepts arc applicable to mechanisms, even that special kind of func- 
tion called “purpose.” The early thesis of llicse authors is likely to 
be accepted today with little if any a^ment in liglit of the wide 
variety of functional propeKica (eg., mcnioiy and learning) com- 
monly attributed to electronic computere. (See Fnank (1919).] 

The significance of this distmetiem for defining will lie in the fact 
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that definitions of stnictural properties arc essentially deterministic 
in character, whereas those of functional properties are essentially 
probabilistic. We shall also find other distinctions within these classes 
of concepts which are significant to defining. 

A complete analysis of the distinction between structure and func- 
tion would take us well beyond the limits of this work. [Se^ or 
example, Singer (1959.] It is necessary, however, at least to sketch 
the major aspects of such an analysis. 

TAc Nature of Structure. Let us begin by assuming an understand- 
ing of spatiotemporal concepts; that is, the concepts of geometry 
(e.g., point and distance) and kinematics (e.g , duration). Let us also 
assume an understanding of the concept of mass (as a measure o 
inertia). It is possible now to construct the concept of a point par- 
ticle which can be identified by its mass and its space-time coordi- 
nates. This is the fundamental element of classical mechanics.* A 
mechanical property of a point particle can be defined as a property 
of such a point which can be expressed as a mathematical function o 
its space-time properties and mass. For example, the average velocity 
of a point particle is exprcs«ed by dividing the length of its travel by 
the time of travel. Its acceleration at a moment of time is the rate 
of change of its velocity at that moment. 

Now a physical body can be conceptualized as a set of point par- 
ticles individuated by its space-time coordinates. The physical prop- 
erties of such bodies are ones which can be expressed completely as 
mathematical fimctions of the mechanical properties of the point par- 
ticles of which it is composed. For example, the temperature of a 
body can be defined as the mean squared velocitj’ of its point par- 
ticles. The mass of the body can be expressed as a simple function 
of the mass of its particles, and its density as its mass per unit 
volume. 

^\Tien we speak of two bodies having the same physical propertyi 
we refer to values of that physical property which are identical within 
some specified degree of variation. For some purposes bodies which 
have a temperature within a thousand degrees of each other may be 
considered the same; in other cases a difTerence of a thousandth of a 
degree may be significant. When we classify by means of physical 
properties, then, we use a range of values along the scales measuring 

• For our purposes here the more fanitiar concepts of classical mechanics (a* 
contrasted with nuantum mechanics) are easier to deal with. A complete tran^ 
latien of this discussion into the coneepta of quantum mechanics, however, i* 
possible: and hence the distiocUtm between these tjrpcs of mechanics is not 06 * 
Dificant in this diacusaen. 
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the properties, a range of Uie researcher’s choosing. A specified range 
on a physical scale specifies what is called a morphological property. 
These properties are tlie typra »e normally deal with in tlie physical 
domain. Morphological proiwrties may he qualitatively treated as 
well as quantitatively; for example, hot, warm, and cold, and heavy, 
medium, and light. 

Now structural propertwa may be defined as the class of geometric, 
kinematic, mechanical, physical, and morphological properties of 
physical objects and, hence, events. 

The A’oture oj Function. In order to make the transition from 
structural to functional properties it is necessary to use tlic producer- 
product re)ation.ship. IVc have considered this relationsfdp in Chap- 
ter 1 and will reconsider it in more detail in Chapter 10. For our 
purposes here we need only review its principal characteristics. 

Members of one class of objects, events, or properties (X) in an 
environment (Fi) are said to be producers of members of a class of 
objects, events, or properUes (F) in an envirooment (F*) jf raerabers 
of X are necessary but not sufficient In £i for the subsequent ap- 
pearance of members of 1’ in Et. Therefore, it no member of X is 
present in Fi, a member of Y will not appear in Et If a member of 
X la present In Ei, ft member of Y may or may not subsequently ap- 
pear in Ei, depending on other conditions. The other necessary con- 
ditions are coprodueers of Y. For example, pushing a button produces 
the ringing of a bell if the bell is not in a vacuum; therefore, the 
presence of air is a coproducer of the bell's ringing. 

The producer and the product may be members of the same class, 
in which case the productive process is called reproduction. Also, Et 
and Ei may be the same environment. 

The relative frequency with which members of X in Et produce 
members of V in Ea is the probability that members of X in Et pro- 
duce members of }' in Et. 

The importance of classification in the producer-product relation- 
ship should be emphasised. Acorns, for example, are producers of 
oaks, but they are not producers of trees because they are not necea- 
sarj’ for all trees (e.g., maples). But acorns are members of a larger 
class of objects which are tree producers. 

Let Jl/i, Ma, • • • represent a set of morphologically distinct objects, 
events, or properties. Suppose that the producere of these things are 
all members of the same morphological class or are the same individ- 
ual. Then the set, Mi, Mj, - • ■ , has Bie common property of having 
been produced by members of the same morphology and hence can be 
said to have a common passive ftmelion. In this sense all the prod- 
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ucts made by a particular company, bo^v'ever different they are m 
morphology, belong to a passive funcUonal class defined as products 
0/ that company. In the same sense the writings of an author, how- 
ever varied their form or content, and all the graduates of a particular 
college would constitute passively functional classes. 

Now consider the case in which each of the morphologically dis- 
tinct elements, Mi, A/2, • • *, has a common product; that is, each of 
them produces objects, events, or properties of a common morphologj’. 
For example, consider an automobile, train, plane, horse and wagon, 
sleigh, and dirigible. These morphologically distinct elements consti- 
tute a functional class by virtue of the fact that they are all producers 
of transport. Similarly, watches, hour glasses, and sundials, which 
have different structures, constitute an active functional class defined 
by a common product, lime telling. 

Now suppose that the elements which constitute a functional class 
are all acta or behavior patterns of (events produced by) a single in- 
dividual. Then that individual can be said to have the function that 
defines the class of its behavior. For example, a person who uses 
matches and mechanical lighters to start fires has the function of 
producing fires. 

An object which has a function by virtue of its membership in a class 
of morphologically different elements which have a common property of 
production is said to have an izUmive function, because possession 
of this properly depends on other things. On the other hand, an 
object which is said to have a function by virtue of its own behavior 
is said to have an intensive function. For example, the time-telling 
function of a watch is an extensive function, but the "fire-starting" 
function of a person is an intensive function because he can start a 
fire in different ways. The watch, of course, can tell time in only 
one way. In order for a thing to have its own function (i.e., an in- 
tensive function) it must be capable of displaying morphologically 
distinct behavior patterns which have a common (morphological) 
product. 

Intensive functions are of two types; nonpurposive and purposive- 
An object has a nonpurposive mtensiTC function if (a) it can display 
only one t>'pe ot behavior in any environment but (6) it can display 
morphologically distinct behavior in different environments, and (c) 
if these different behaviors have a common product. For example, a 
ship’s automatic pilot will behave differently for each different devia- 
tion of the ship from a specified course. But its behavior in each of 
these instances has the same product; bringing the ship back on 
course. Similarly a producUon^rontrol device which maintains a proc- 
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pss at a constant temperature despite environmenta) changes also dis- 
plays a nonpurposive intensive function. 

A purposive intensive function, or simply a purpose, is displayed 
by an object if tlie following condiUons are satisfied: tie object can 
display morpliologicalJy different behavior patterns in a single en- 
vironment, and these behaviors have a common product. The pur- 
pose is defined by the common proitcrty of the outcome. For exam- 
ple, suppose tiiat we observe a'periidii wnoTn me s^e environment 
at different times nails, screws, or glues wood together. We r.an sny 
that his behavior is purposeful and his purpose is to join wood to- 
gether. 

Purposive behavior is not restricted to human beings or even living 
ihin^. It is now quite commonplace to program computers so that 
they disph^y purposive behavior (eg., in a game setting), the purpose 
being to win the game. In this program the computer has different 
courses of action available to it in each situation, and although prob- 
abilities can be associated witli each of its moves even the pro- 
grammer cannot predict with certainty which move it will make. 

In purposive behavior the alternative courses of action arc fre- 
quently called trifORS, and the coromoo property of the outcome is 
called an tnd or ob/ective. 

If the dlstincti<ms between the types of functions are kept In mind, 
then the definer ean, by classifying Uie functional concept with which 
tie is dealing, establish some requirements of its definition. For ex- 
ample, suppose that wc want to define the property ‘'aggressiyfBfiss." 
Without knowing precisely what this concept means, it seems clear 
that it refers to a property tJiat we attribute only to objects capable 
of displaying purposeful behavior. In defining this concept, then, 
we must specify the class of behavior to be observed and the circum- 
stances under which it will be observed. On common-sense level 
wo would say that a person is aggressive if he enters into conflict with 
otJiers ‘witfiout provocation. Wc can make this notion more precise 
as follows: 



(а) when he shares an environment wito another individual or in- 
dividuals, B; and 

(б) B’s behavior has no effect on the efficiency with which A ean 
pursue his desired objectives; and 

(c) A has equally efficient alternative courses of action which fall 
into one of two classes: (i) those which reduce B’s efficiency for his 
objectives, and (ii) those which do not, 
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A has a greater probability of selectiBg courses of action of the 
first class (ij than of the second (ii). The degree of A's aggressive- 
ness can be defined as the probability that he chooses a means of 
type (i). (Aggressiveness with provocation is generally called ascend* 
ancy.) 

We turn now to a more detailed consideration of formal require- 
ments on operational definiUons of structural and functional prop- 
erties. 

DeBnib'onal forms of structural properties 

Our interest in structural properties may be of three types: (1) in 
the property of a body at a moment of time, (2) in the change or 
rate of change of such a property under constant conditions, and (3) 
in the change or rate of change of such a property under changing 
conditions. 

An operational definition of a structural property of a body at a 
moment of time should contain reification of 

(a) the object or class of objecU to be observed, 

(b) the conditions (environment) under which the observations 
should be made, 

(c) the operations, if any, which should be performed in that en- 
vironment, 

(d) the instruments, if any, and the metric standards which are 
required to perform the specified operations, and 

(e) the observation (s) which should be made. 

Suppose, for example, that we want to define what is meant by the 
assertion that a particular object “is red.” We might proceed as fol- 
lows. (q) We identify the object whose color is to be determined, 
(fc) We specify the environment in which its color is to be deter- 
mined; for example, the abnosphcric conditions, temperature, and 
lighting conditions. These conditions are determined by the pur- 
poses of the inquiry, (c and d) We specify where the object should 
be located in the environment, and what instruments (e.g., spectro- 
scope) should be used and bow. (e) We designate the spectral range 
of wave lengths (say 0.00006 to 0.000(B cm) into nhich the reflected 
light should fall. 

The second type of definition involves a change in a structural 
property over time in a constant oivironment. Such a definition re- 
quires the same five tj-pes of specification as in the preceding case 
plus a specification of the lime interval over which the observations 
are to be made, the timing of the obsen-ations, and the way in which 
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the data obtained are to bo treated. “Rate of dissolution,” for ex- 
ample, is sucl) a properly. It involvca the Jenjfth of time required 
for an object to change certain of its alruetiiral (e g., chemical) prop- 
erties under constant eoniliCions (eg., while immcrseii in a specified 
liquid). ‘‘Rate of deformation” of a structural member of a building 
under constant load is a sunilnr property The so-called life ■proper- 
lies of gootls, tools, and ntuipment fall uila this class of structural 
properties. Thu life of a lamp bulb, for example, might be defined 
as the lengtti of time it emits light in a specified constant environ- 
ment. 

The lliird type of structural property involves changes of a prop- 
erty under changing conditions The form of this definition is simi- 
lar to the prert'ding one with the additional requirement for specifi- 
cation of vhat changes in Uie environment should be made and how 
they should be timed. Observations always involve reactions to these 
changes. The coefiicient of linear expansion of an object is an ex- 
ample of such a property, as is the coeflicient of volume expansion and 
the coefficient of compressibility. So-called scnsitii'il-y properties ail 
fall into this class as well. The sensitivity of photographic paper to 
light, of an explosive to impact or heat, of a structure to shook, and 
so on, can all be defined n-itldn the form de.*enbed. 

ZTefioitfonal forms of functional properties 

Since the meaning of function is rooted in the meaning of tlie pro- 
ducer-product relationship, it is not surprising tliat the essence of all 
functional concepts lie* in a measure of probjibility of production. 
This prob-ability may be of either of llie following types; 

(1) The probability that an individual object or group will select 
a specified course of action. 

(2) The probability tiiat a s|)ccified eoursc of action will produce 
a specified outrome. 

These probabilities correspond roughly to measures of preference and 
efficiency. The measure of every fiinetional property reduces to a 
measure of one or botli of tlic«s types of probability. For example, 
in the concept of aggressiveness discussed above, the degree of ag- 
gressiveness was defined a** tlic probability that an individual would 
select a type of action which reduces the efficiency of a co-occupant 
of his environment. 

Functional properties arc of three general types, corresponding to 
the types of structural properties already considered: 
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(fl) the property of something at a moment of time, 

(h) tlie change or rate of change in a properly umlcr constant con- 
ditions, and 

(c) the change or rate of cliangc in tlie property under changing 
conditions. 

Definitions of Use first type should contain specification of 

(1) tiie object or class of objects to be obscr\-cd, 

(2) the conditions (environment) under which the oi)«er\'ations 
should be made, 

(3) the operations, if any, which should be performed in that en- 
vironment, 

(4) the instruments, if any, which arc required to perform the 
specified operations, 

(5) tiie observations which should be made, and 

(6) the treatment of the data obtained. 

This is quite similar to the form of the definition of the correspond- 
ing type of structural property. If no observational error is involved 
in the determination of a structural property, only one observation 
need be made. In the ease of the functional property, however, even 
where no obser\'ational error Is present, an Infinite number of ob* 
sen'atloDs are required (in principle) in onlcr to determine the ap* 
propriate probabilities without error. 

An example of such a property is "the degree of familiarity of an 
indmdual with a means to an end." To define this concept we should 
first identify the individual and objective (end) involved. Kext we 
should specify the conditions under which the observations should be 
made. These are; 

(a) A set of alternative courses of action (Cj, Cs, avail- 

able in the environment. 

(b) Each of the available courses of action has perfect efficiency 
for the specified objective. 

(c) The individual has interest in only the specified objective. 

In this situation we should observe the frequency with which the in- 
dividual selects each course of action. Then the degree of famiharitj' 
of the individual for a specified couree of action, C<, relative to the 
objective in that environment is Uie probability (limiting relative fre- 
quency) of his selecting that course. 

As can be seen from this defimtion, "familiarity” is a preference- 
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type property. All funcUtmal properties relating to preferences should 
have definitions of the form indicated. The same is true for "disposi- 
tional" properties; for example, hungry, tired, and bored. 

The second type of functional property is similar to the first except 
that out concern is with changes in probabilities under constant con- 
ditions over time. These properties are analogous to such structural 
properties as solubility or rale of deformation under constant load. 
People, for example, become tired of certain things after a while, or 
else may become increasing!}' fond of them. This simply means that 
their preference patterns change over time. The same may be true 
of, say, an inspection machine whose probability of rejecting an ac- 
ceptable item may change with use of the machine. Another set of 
properties of this type involves changes in the rate of performance of 
a task with its repetition. 

The definitional form of such a property is similar to the first ex- 
cept that the way of measuring the change in the relevant property 
must be specified, as must the time interval to bo covered and tbo 
frequency or timing of tlie obsen-ations. For example, one could, 
measure the change of degree of familiarity with a course of action 
as the change in this degree between two moments of lime, ii and it; 
or one could measure the average rate of change of this probability 
measure will) respect to time (i.e., the average derivative with respect 
to time). 

The third and last class of functional properties to be constdeted 
involves probability of choice or cfiicicncy under changing conditions 
and, hence, parallels the third type of structural property considered: 
sensitivity properties. The conesponding functional properties arc 
called sensibility properties, because they involve sensation, that is, 
functionally distinct responses lo stimulation. In the case of sensi- 
tivity we were concerned with the variations in stimulation neccssarj’ 
to produce certain changes in structural properties. Here we arc 
concerned with the variations in stimulation required to produce cer- 
tain changes in functional properties. The stimuli may themselves be 
structural or functional in nature. Sensibility to noise, for example, 
would involve structurally defined stimuli; whereas sensibility to ag- 
gressiveness would involve funetionally defined stimuli (i.c., the meas- 
urement of aggressiveness 5s made under conditions where other peo- 
ple’s bchaidor Is defined functionally). 

The definitional form of this t}'pc of property, Ihcn, is similar to 
the second except that it is nccewarj' to specify the stimulus and the 
operations by which it must be "ndminislCTcd.” 
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AN EXAMPLE OF DEFINING 

To illustrate the various points which have been made about the 
content and form of definitions and to demonstrate the roapitude o 
the task of defining we will consider in some detail a functional con- 
cept which has been receiving increasing attention in scientific litera- 
ture, information. As wc shall sec, the amount of information con- 
tained in a message will be defined in terms of changes in probabili- 
ties of ehoicc in the recipient of the message (a stimulus). 

As in most definitional efforts, some of the concepts which must be 
used to define information themselves require definition. Here wc 
shall merely identify these prempposed concepts, hut the reader can 
find an extended discussion of their meaning in Ackoff (195S). 

These concepts are: 

/: an indi>’idual or entity to which purposefulness is to be at- 
tributed. 

C«; a course of aelion; i m. . ^ 

Oji a possible outcome or consequence of a course of action; 
1 ^ j ^ n. 

Pi', the probability that I will select Ci in a specified en^•ironment, 
J^;thati8,P» = P(C,17,.Y). . 

Eijt the probability that 0/ will occur if Ci is selected by 7 m A i 
that is, the efGcienc)* of Ct for Of in A'. 

7?./ = P(0/|C., 7, AT) 

Vf: the value (importance or uUlily) of Of to /. 

An indmdual (7) may be said to be in a purposeful state in an en- 
vironment (AT) if the following conditions hold: 

(1) There are at least two exclusively defined courses of action 
available in A'; that is, 

(2) For at least two of the available courses of action in N , the 
indi\ndual's probability of ehcnce is greater than zero; these are called 
potential courses of action. 

(3) Of the set of outcomes (so defined as to be exclusive and ex- 
haustive) there is one (say 0,) for which two of the potential courses 
of action (say Cj and Cj) have some efficiency; that is, Eu > 0 
Eja > 0. Furthermore, Ei«^Es,. 

(4) The outcome relative to which condition (3) holds has some 
value to 7; that is, F,^0. 
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This {Jefinition may be summarized les<j technically as follows: an 
individual may be said to be in n purposeful state if he wants some- 
thing and has unequally efficient alternative ways of trying to get it. 

If we consider an individual over a period of time it will be con- 
venient to refer to tlie purposeful states at the beginning and the end 
of that period as initial and (ermtnaf states, respecUvely. 

Now we can turn directly to the meaning of informaCion. 

Shannon, his predecessor, R. V. L. Hartley (192S), and most others 
who discuss information in mathematical terms are concerned with 
the amount of information * that can be communicated rather than 
with the amount that is actually communicated. Shannon was pri- 
m.'triJy involred with sj'stems in whicJi cacii possible message can be 
codeil into combinations of sj’mboU For example, if there are four 
po&slhle messages and two discrete symbols (0 and 1), the messages 
can be rcprwenteil as 00, 01, 10, and 11. Then, to select one message 
out of the four, two choices from among U»e two symbols (i.e., binary 
choieca} may be made. One binary choice alloTvs two messages (0 
and 1), and three binary choices allow eight messages (000, OOl, 010, 
100, no, 101, on, and inj. In generat, x binary choices allow 2* 
possible messages. 

For Shannon, the amount of information eoDtained in a message is 
Uio amount of freedom of choice i»%-olved 3d the selection of the 
message. A unit of choice is defined as the selection of one out of 
two equally available symbols. Thus, in selecting one of two equally 
available symbols, one choice unit is involved and the resulting one- 
symbol message contains one unit of information. 

In general, if there arc jlf equally a^'aiIable messages in a state, the 
selection of one contains x units of information, where 

a: = logs M. (1) 

Equal availability of the symbols means equal likelihood of choice 
by llie sender. That is, if there are if possible niessagea and the 
probability of each being selected is 1/Jlf, complete freedom of choice 
exists. If the probability of scfecting s particular message, P(, devi- 

•Because of the pervasirencas of tbc use of “iofonnation" in ShaaBoaa re- 
stricted sense, it might secni preferable to use another term here. But since the 
usage here eonforms more cloaely to eoramoa usage, if a ebaago is reqnired, it 
would seem preferable to change Sbannon’s lerm. As Colin Cherry (1S57, p. 60) 
Botes, "The measrrre for /T, Uvmge jftforiBslJonf from Wiener and ffhannoo ih 
npplieable lo Ihc signs themselves, and does not concern their meaning, In a 
sense. It is n pjty that llie malhemslJrtl concepts stemming from Hartley have 
been called 'information' at all." 
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ates from l/M, there is not a completely free choice. In the exj.rcine 
case, if the probability of selecting any one of a set of messages is . . 
then there is no freedom of choice and no information can be com- 
municated by the one message which is always eelected. 

In order to cover cases in which choices arc not equally likely (as 
well as where they are) , Shannon derived the following general meas 
ure of the amount of information (symbolized by II in his sj'stcm) 
contained in a state: 

7/ = -SPJogA,* 

where P« is the probability of choice of the tlli message. If 1^ 
used, then II is expressed in binary* units whicli arc called bits, 
a state which contains two equally likely messages contains one bi 
of information. 

The measure of information to be developed here will also be re- 
lated to freedom of choice; that is, it will be a function of the prob- 
ablliUes of choice associated with the alternative courses of action 
available to the receiver rather than with the alternative messages 
available to the sender. The measure here will also be a function of 
the number of alternative potential courses of action, m. 

When we talk of the amount of information that a person has m a 
specified situation (state), we do so in two different but related sensei 
First, we refer to the number of available courses of action of whw 
he is aware; that is, to the number of potential courses of action. 
For example, a person who is aware of four exits from a particular 
building has more information Uian the person who is aware of only 
two when there are four. TJ»e act of informing, then, can consist o 
converting available choices into potential choices. For example, a 
statement such as, “There arc exits at either end of this hall,” may 
convey information in this sense. The person who has this informa- 
tion (i e., who has these potential choices) may or may not exercise 
it, depending on his appraisal of the relative efficiencies of the alter- 
native exits. In one sense, then, infonnation is the amount of po” 
tential choice of courses of action which an individual has. 

The second sense in which we talk of information involves the basis 
of choice from among the alternative potential courses of action. For 
example, an individual who knows which exit is nearer than the others 
has a basis for choice and hence has infonnation about the exits. Iii' 

•For an edifymg discuBsiMi of this incasure and its derivation tee Cherry 
(1937, pp. m-so). 
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formation in this sense pertains to the beliered eflieiencies of the aJter- 
natives relative to desired outcomes (eg, a rapid exodus). Suppose, 
for o.xample, Ihrtl there are tno exits and one is nearer than the other. 
If this is known and tlic objective (valued outcome) is to leave the 
building quickly, tiic choice is detenmned in the sense that the indi- 
vidual will always select the nearer exit. If he always selects the 
more distant exit, tlicn he is obviously misinfonned (i.e., he has in- 
formation, hut it is incorrect). If he selects each exit with equal 
frequency, then ho apparently has no basis for choice; Hint is, no in- 
formation about tlic e.vits. Jn Ibis sense, then, information is the 
amount of choice which has been made. Now let us define and de- 
velop a measure for information In this second sense. 

Consider the case of an individual (/) who is confronted by two 
polenUal courses of action, Ct and C*. If the probabilities of select- 
ing the courses of action arc equal, Tj « P* = the situation may 
bo said to be tndetcrmmate for I. He has no basis for choice and 
hence can be said to have no ioformation about the alternatives. This 
Is clearly the case when one of the alternatives is more efficient than 
the other. Cut If the tno courses of action arc equally efficient, the 
Individual may have inlormation to this effect and select each with 
equal frequency. Strictly speaking, however, he has no objective 
clioico in this situation, since the alternatives arc equally efficient. In 
n situation like this— a nonpurposcful state— information has no oper- 
ational meaning. Consequently this discussion lias relevance only to 
situations in which nil the nUemative courses of action arc not equally 
efficient 

If P, z= 1,0 and Pa = 0, then the situation is determinate for /; all 
the choice tliat can be made has been made. The ra-aximuni amount 
of possible Information is contained in the state. It may not be 
correct, but this is another matter which will be considered below. 

We may define a unit of information ns the amount contained in 
a two-choiec situation that is determinate. 

Let us consider Uie general case involving ?« alternative potential 
courses of action. In onler to select one from this set, a minimum of 
(w — 1) choices from pairs of nllematives is required. Table 5,1 il- 
lustrates tills fact. 

We can conceive of the amount of information contained in a pur- 
poseful stale, then, as a point on a scale bounded at the lower end 
by indeterminism (i.c., no clioico has been made) and at the upper 
end by determinism (i.c., complete choice has been made). Location 
on this scale will depend on the values of Pi. 
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Table 5.1 


2 3 4 5 



In an indeterminate state each Pi = l/w- Therefore, one measure 
of the distance of a state from indeterminism is 


E p.-z - 


( 3 ) 


For an indeterminate slate this sum is equal to zero. In a deter* 
miaale state one P< is equal to 1.0, and the remaioisg (m — l)P/8 are 
equal to zero. Therefore, in a determinate state, 


Z P. 


m I \ tn/ 


+ («-!) 0 -- 




.-i = 2--- (4) 


The ratio of (a) the deviation of a specified state from on inde- 
terminate state to (h) the deviation of a corresponding determinate 
state from that indeterminate state, then, provides a measure of the 
fraction of the ma-tiroum infomiatioa such a state can contain, to that 
which it docs contain. S>'mt>oIically this ratio is 



in 


(« 
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The prtxlucl of lliU fraction and the maximum amount of infonnotion 
each n state can contain— that is, (m - 1)— provides a measure of the 
amount of information (here symbolued hy ^1) in that state: 



Now llic amount of information communicated may be said to be 
the difference between tlic amounts of information contained in the 
slate of the reech'cr immediately preceding the eommuoicstion (i.e., 
the initial state) and in tlie state immediately following the communi- 
cation (i.e., the terminal state). Let /i(S|) be the amount of infor- 
mation in the initial state, and ylbSj) tlie amount of information in 
the terminal state; then the amount of information communicated, 
At, is given by the following equation; 

(?) 

which may also be ^7r^t(cn In an expanded form: 



where w is the number of potential coursca of action in the initial 
slate, in' is the number of such choices in the terminal state, and Pi 
and Pi arc the probabilities of choice in the initial and terminal states, 
respectively, 

d, can take on values from — (w — 1) to (m' -f- 1). Negative val- 
ues represent a loss of information (e.g, as in going from a deter- 
minate to an indeterminate state). 

The concept of negative information may seem ohiectionable or at 
least strange. But on reflection tins strangeness disappears. Clearly, 
information can be discarded or lost by forgetting it. A message may 
mdeTOotic a jwfswi’s acceptiwee at a piece at Wamitiou. awi hewie 
induce him to discard it. In briunwaaliing we have such communi- 
cation in a concentrated form. 

It should be noted that this measure contains no Iroplicalion con- 
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cerning the correctness or incorrectness of the information received 
Further, it should be noted that this measure is relative to » 
receiver in a specific state. The same message may convey diSmn 
amounts of information to different individuals in the same state or 
to the same individual in different states. Consequently, to specify 
the amount of information contained in a message it is necessary 
specify the set of individuals and states relative to which the 
is to be made. If more than one individual or state is involved, i >3 
also necessary to specify what staUstic (e^., an average) is to be use . 
Generality of information may be defined in terms of the range o 
individuals and/or states over which it operates. 

It should also be noted that messages are not the only source o 
information. One may obtain information by observation. For ex 
ample, one may count the number of exits from a house. The measure 
of information suggested liere is applicable to information obtaine 
by cither observation or communication. 

In the definitional effort just completed we have fused both con- 
ceptual and operational defining. The concept of “information 
derived from other concepts whose meaning is, in general, clearer than 
that of “information,” and, at the same time, the ground work was 
laid for an operational definition of the concept. 


CraCULAmTY OF DEHNINC 

As is apparent in the definition of tn/orniafion, each definition must 
resort to other concepts. These may be as complex as the concep 
being defined; hence the search for ultimately simple concepts y 
Stevens and others. But simplicity is relative, not absolute as Stevens 
supposes. It may be very simple to determine whether an object is 
red where the consequences of error are trivial. But if the observers 
life depends on the color determination, the problem becomes as com- 
plicated as possible. 

Tlie denial of ultimately simple concepts seems to imply that a 
scientific defining is circutar. In a sense this is true, but defining 
takes place in three dimensions, not two. ^lien a full circle has b«n 
made, wc are above our starting point and have brought to it nen 
and more precise meaning than it had when we started In cnee , 
when we define one concept, this definition illuminates the concepts on 
which it depends as well as the concepts which depend on it 
In order to demonstrate this point, in Psychologistics (1947) an 
subsequent writings, Churchman and Ackoff began with concepts o 
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mechanics and derived the iirincipal concepts of pjiysics, biofogy, psj’- 
chology, and Uie social sciences. Then, using the results, they turned 
back on the scientific circle and defined tlie concepts of logic, mathc- 
rnatics, and mechanics in terms of psychological and social con- 
cepts. The meanings of llic mcchanicnl concepts at the end of this 
cycle were considerably enriched by the process. To illustrate this 
point Jet us consider the concepts of temporal precedence, "before" 
and "after." These liave generally been considered to be extremely 
difficult, if at ail possible, to define. Wc find a typical reaction to tins 
difficulty expressed by Albert Dnstein (1923, p. 1) : 

The experience of aa indivUiwI apficars to lie arrnngMl in a series of 
events; in this aeries the single cxtnts ue rcmaciher appear to be onlereil 
aeconling to the criterion of •'earlier" and ■'later,” which cannot he analyred 
further. There exists, therefore, for the individa'il, an f-time, or subjective 
time. Tliis in itself is not moasoraWe. 

The analysis of tlie “unanolytablc" ami the conception of the "in* 
conceivable" have in the past constituted some of the mo«t important 
spurts in the progress of science. When concepts are widely regarded 
as indefinable even by great scientists, this may he an indication of 
the need for some very basic reorientation in the way these concepts 
are regarded. Tlie reorienUtiou in Ibc way of looking at "before" 
and "after" here consists of regarding these concepts ns funetional 
rather than structural; that is, of considering tliem in terms of human 
purposes. 

An interesting and important early attempt to define tliese concepts 
in terms of purpo« was made by Charles I’circe |in Buchler (1040)). 
For Peirce "future conduct is the only conduct that is subject to self- 
control" (p. 2C3). TJiis 8Ugge«feil a criterion (p. 313) for distinguisb- 
ing the past from tlie future; 

One of the most marked features about the law of mind is that it makes 
time to have a deCnito direction of flow from past to future. The relation 
of p.tst to future is, in rdetence to the J.iw of mind, different from llie rela- 
tion of future to past. Ttus makes one of the great contrasts between the 
law of mind and the l.iw of physical force; there is no more distinction be- 
tween the two opposite directions in tnne than between moving northward 
and moving southward. 

In order, therefore, to analyze tie law of mind, we must begin by asking 
what the flow of lime conasts in. Now, we fiml that in reference to any 
individual state of feeling, all others are of two classes, those whicli affect 
this one (or have a tendency lo sficcl it . . .) and those whidi do not. The 
present is affecliljle by the past bnt not by the future. . . . 

If, of two states, eaeh is absolutely uaaffecUUo by the other, they are to 
be regarded as parts of the same state. They are contemporaneous. 
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Roughly, then, Peirce would eay that one eUtc prcccdctl another 
for a PCRon if he treats one as potentially afTccling the otlicr, but b 
vice versa. This is an insight which we want to employ in defining 
the direction of time, though Uicrc are sea-cral details * "“f® 

to avoid getting involved in a vicious circle. Tlic insight is that p - 
posivcly endowed individuals distinguish Uic past from Uie future on tne 
basis of what they think they can control. With respect to the future, 
there arc always things that their present behavior may accomplwn, 
but they do not regard their present beha«or as having anj jictcn la 
influence on the past. Even a roan in a prison cell, or a coinplc e 
paralytic, thinks of himself as capable of changing the future in some 
respect, even though it is only a change in his own behavior: rom 
silting to walking, or a change of a recollected image. 

To refine the conceptualization of the future as that which a pur 
posivc individual takes to be polenUally alicctible by his behavior, we 
must give a more accurate account of the concepts involved. >-Uci 
terms as affect and infiuenee arc inappropriate for our present pur- 
poses, for they arc equivalent to the notion of "produce,” which a 
ready contains the notion of the future. It will be recalled ^ 
defining tlie relation ‘‘.Y produces V," one of the conditions was tha 
X precede V. Hence, if we d'lstinguish the future from the past in 
terms of potential production, we Involve ourselves in a fruiUcM, an 
hence a vicious, circle: objects existing nt one moment of time *1 
are potentially producible by those of a past moment (», but this is 
so simply because "potential produclion" immediately entails the no- 


tion that (i comes after f«. 

To avoid this diflicully, we introduce U»c concept of cijcnfmfity' * 
is essential for V it A' is necessary but not sufficient for T; no refer- 
ence to precedence is made in this definition: nothing is said as 0 
whether or not X precedes V in time. Of course, it may seem 
doxical to say that, if X comes after I' it can be essential for I , hu^ 
actually such usage is possible. For example, not only can we 8 a> 
that an explosion yesterday was necessarj’ for the ruins we see today, 
but we could also say that the absence of ruins today is essential for 
there not having been an explosion (a nonexplosion) at a certain place 


yesterday. 

Now, purposive individuals regard some objects and behavior ^ 
potentially essential for other objects and events, and do not regard 
them in this light with respect to other objects and events. If 
event takes place at some distant place, then we may not regard our 
present behavior as potentially essential for that event. If there i* 
one property of an event that is (virtually) sufficient to prevent a 
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person from reearding an event X as potentially essential for an 
event Y, this property is one of precedence (i.e., 1’ precedes X). 

Put another way, when a person conriders the past, he considers it 
deterministically; that is, the past is connected (for him) with the 
present by deterministic causal Ian’s. As a result, he does not respond 
to tlie existence of alternative behavior patterns which can be consid- 
ered as potentially «scntial for the past. If he regarded the future 
in the same deterministic fashion, then there would be no alterna- 
tives with respect to it either. Only one mechanically specified act 
in tlie present will completely determine a specified mechanical state 
later. And here is the crux of the matter. A person generally does 
not consider the future deterministically. He specifics at least some 
future states morphologically or functionally, and hence he can re- 
late- the present to the fultire only by necessary but not sufficient con- 
ditions (producers), and he considers alternatives. It is to be noted 
that, in regarding something as potentially essential for something 
else, we take it as a nienrber of a collection of things which are Mi 
mechanically identical, and wc regard it as existing in one of a col- 
lection of environments. In other words, our attitude toward the fu- 
ture is to regard the future process as one of a collection of processea, 
in tome of which current objects and behaviors are essential for future 
events. We try to choose that behavior which is most likely to be 
essential for some intended goal of the future. But with respect to 
the past, wc do not take present objects and events to be essential; 
that is, we do not respond to whatever essentiality they may have 
with respect to the past. 

Hence, we can use the notion of potential essentiality as a basis for 
defining before and after as follows; consider two states of nature 
designated by the momentary (temporal) properties ti and t#. If a 
person virtually always fails to regard any object of ti as potentially 
essential for it, but does ivgnrd some things in as essential for some- 
things in li, then for him t» precedes ti. 

This, then, defines what Einstein (above) called subjective time. 
Objective time is something else. It refers not to how the individual 
responds to the content of two states, hut to the effectiveness of his 
response relative to his objeclires; that Is, to whether or not what he 
docs in f» does in fact give him some control over the properties of 
tile state at fi. 

It should be noted lliat even in the ca.«e of subjective time it is 
nssHiiKxl that the subject (as well ns any ob»crver of the subject) can 
distinguish between two states and hence between two moments of 
time. He may be able to do this, how-ever, without knowing which 
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one precedes the other, as, for example, when he forgets which of two 
clearly distinguishable past events preceded the other. 

The reader will obsenx how difficult it is for him to think of the 
past in nondeterministic terms; that is, to conceive of the past in 
such a way that alternative essential conditions for past states occur 
to him. This difficulty is to be expected in light of the definition 
given, since, in effect, tlic past is defined in terms of our inability to 
conceive of alternatives with respect to it. 

Since science has made few efforts to define structural concepts in 
functional terms, it is only natural that a definition such as has just 
been developed seems awkward or even forced. But with e.vpcriccce 
in these matters such definitions will eventually be no more strange 
than the use of structural concepts In defining a functional concept 
is currently. 


SUMMAIiy 

Defining is an aspect of the research process which all too few scien- 
tists take very seriously. The meanings of coneepta are too often 
taken lor granted. Yet definitions arc essential as criteria for rele- 
vance of data used in evaluating variables and constants in all typ<^ 
of scientific statements: theories, laws, facte, and decision models. 
Such criteria can best be provided by idcalired operational definitions 
which specify the conditions under which and operations by which 
questions concerning the concept involved "ought" to be answered. 
The "ought” represents the best that we currenUy know. Such oper- 
ational definitions provide research standards which serve as a basis 
for adjusting data obtained under less than standard conditions. In 
adjusting data, however, the dcfiaftlons must be supplemented by 
necessarj’ theory and laws. 

The content of (or meaning contained in) a definition should take 
into account the objectives of the researcher and common and scien- 
tific usage of the concept. The former is necessary if the definition 
is to pro\ide a standard of accuracy required by the inquiiy. The 
latter is necessary if the objective of communicability of results 
to be obtained. 

Objects are defined by specifying a set of properties which are suf- 
ficient for inclusion in the class of interest. Events are defined by 
specifying a set of changes in properties of dcsignateil objects The 
definitions of both objects and events, then, depend on the meaning 
of properties. 
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The formal rcquirctiicnU for (operationally) defining properties are 
(hat we epeeify what is (o be obsWTrcfl, under wJiat (changing and un- 
changing) condilions the obacr^’alions arc to he made, what opera- 
tions arc to ho perfonned, what instruments and measures are to he 
lived, nnd how the ol/rervations are to l/e made and treated. 

A fundamental distmction was draw'n between struclurnl and func- 
tional properties. Tlie foraier refer to the material of which objects 
arc made and their form. TheJatIcr refer to the origin or use to which 
objeeta can be put or pul themsclvc-s. The principal distinction be- 
tween these t>*pe8 of concepts from the point of view of defining is 
that functional projicrlica all involve mcaaurca of probability, either 
probability of choice or probability that a course of action will pro- 
duce a siwcificif outcome. For alnicturnf properties the measures in- 
volved are ideally deterministic. 

All definUiona use other concepts and hence presuppose their mean- 
ing. In general we try to use simpler concepts than the one being 
defined. Dut simplicity is a relative concept, not ttb«olutc. There 
arc no concepts that arc absolutely simple and whose meaning is fixed 
and known. TIds docs not imply that the meaning of some primitive 
rancepts must be taken for granted and remain undefined. Every 
scientific concept is poteulially definable. Tliis circularity of mean- 
ing need not bo vicious because in the process of rounding the eireJe 
the meaning of initial concepts can be considerably enriched. 

The meanings of concepts evolve witli use and changes in the con- 
ical in which they are used Thej- ore nl«o modifieil by the discoverj’ 
of interconnections between concepts which arc made explicit by try- 
ing to round the definitional nrcfc. The progress of science, pure and 
applietl, is as dependent on progress In defining ns nn progress in any 
other aspect of inquiry. 
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6 

MEASUREMENT 


INTRODUCTION 

Scientific defining and loeasurcmcnt arc very closely related, aa 
Peter Caws (1959, pp. i-l) bat observed: 

Definition and tnensvrement certaioly ba^o hmetiona) sunilaritics nlilch 
mabc it almost inevitabfe that a discussion of one abouid tooner or later in* 
Yolve tbo other. They both have the character of leading to relations which 
set the entities of science in order with respect to one another. The kind 
of order tkit they establish can be broadly differentiated, but they run to> 
eetber in many coses, so that there are times when the two procedures seem 
to amount almost to the same thing. DeTioition, in general, b concerned 
nitb the systematic order of the conceptual scheme of science, and with the 
nature of the relations between different entities. Measurement has a more 
limited function, that of establishing metrical order among different manU 
festations of parUrulnr properties, and of making scientific events amenable 
to mailicmalical description. Often the relation between two dijerent prop^ 
erties is not cleat unless measurements have been carried out on both in 
some rase where they appear tc^ether; nowadays much definition is expressed 
in ft mathematical form which presupposes measurement. 

This discussion by Caws shows the dose relationship between meas> 
urement and definition, but it does not make clear the precise nature 
of this relationship. Measwement will be used Iiere to designate the 
priTceduro by which we obtain ^nnbols which can be used to represent 
the concept to be defined. Tire scientific definition of a concept tells 
us under what conditions what obaerraliona should be made, ^teas- 
nr 
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urcment is involved in that part of the definition which eoricenu it- 
self with how the obsen-ations should be made and treated so that 
the concept defined may be represented by sj-nibols with certain 
properties the specification of which is the subject of this chap r. 
Therefore, measurement is here taken to be an essential pnrf of scien- 
tific defining, but not all of it _ .. 

A sj'mbolic representation of the concept defined is not ncccs^n y 
measurement. The property of the symbol most usually cited as being 
necessary', it not sufficient, to make it a measurement is that it be 
or contain a number. But, despite its intuitive appeal, as wc sia 
see, this is neither a necessary nor a sufficient condition of measure- 


racni. , 

The term anf/imefiration is sometimes applied to any procedure 
of assigning numbers to objects, events, or properties. In common 
usage measurement is restricted to processes which involve use o a 
constant unit of measureroent. In science, however, it has become 
increasingly common to apply the term meosuTement to any process 
which involves “the assignment of numerals to objects or events ac- 
cording to rules” [Stevens (1946, p. 677)], or "the assignment o 
numerals to represent properties” (Campbell (1057, p. 267)]. 

Such a definition of measurement is clearly incomplete, if not m* 
correct. For example, we can establish rules for assigning random 
numbers to objects, events, or properties of these, as we cemmonlj 
do in random sampling. The obvious objection to such a procedure 
is that on successive applications it docs not yield the same, or approxi- 
mately the same, numbers to the same things. It is quite simple, how- 
ever, to modify the procedure so that the same random number is 


always assigned to the same thing. 

The purpose of measurement is to represent the content of obser- 
vations by symbols which arc related to each other in the same way 
that the observed objects, events, or properties arc or can be. ?'uro- 
bers are the symbols generally used because their interrelationships 
have been exhaustively studied in mathematics, and because wtoe 
of the relationships between them are shared by the observations- 
But since numbers do have properties which the observations do 
have, it is important to make explicit which of their properties are an 
are not shared by the observations. We shall have more to say on this 


subject below. 

In grading a set of exanunation papers we may express the grades 
either as numbers (e g., from 0 to 100) as letters (e.g , A, B, C), 


as words (e g., excellent, good, fair). There is usually a way of trans- 
fonning from one set of these symbols to another. The same choice 
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of eymboh is possible in inetsnrine any property. Tlio use of a letter 
or a word is no Jess me.nsurcincnt tlian is tJic use of a number pro- 
\*id«l that wc make oxpUeil, as wc must in the case of numbers as 
well, what operations may be performed on the symbols. Clearly, if 
wc want to manipuhatc tlic sy’roboJs, numbers are the c-asicst to use, 
but rules can be get down for ]ierfonning the K|uiviilcnt operations 
on otficr types of sj-mbols. If they ore, then (Jierc is no liiffcrence in 
the uses to which these gymboJs and numbers can be put. 

We can spend an indefinite amount of time trying to specify a sot 
of ojiorations which define ineasurcmcnL History has shonui that sucli 
cftoits arc fruiUcM, since the operations ol masureincnt are changing 
ami developing firogressiieJy as arc all other types al scientific opera* 
lion. But wc can look at measurement anoliicr way. Wc can define 
it as a jiroress whose output can be usctl in a particular way. This 
makes the cascnlial property of measurement the functional properties 
of its protluet, rather than the way It is produced, 

Afl we fi.ive indicated, the produeta of roeasureroeot arc symbol?, 
know ledge of w hich is erjuivalent to knowledge of the properties reii- 
resented. For example, to know tJiat a c.artoa is 3 fcct-0 inch wide 
and tiiat ii doorway is 3 fect-C inches wide ia to know that the carton 
will pass through the doorway. MeasuremenU nllow us to compare 
the same properties of different Udngs, and the same property of the 
same thing at difTcrcat limes, and to describe how properties of the 
same «r different things arc related to each other, Therefore, in 
gcnenil lenns, measurement can be dcBnetl in terms of its function: 

It M a tfo!/ of oblamny fi/m(io(s to represent lAe properfies of o6;ce<s, 
events, or state*, ichkh si/mbob hove the same relevant reJotionship 
to each other os do the things which arc represented. 

In this chapter wc will conccnlrale primarily on tlie use of numbers 
in measurement. But the reader should not lose sight of the fact that 
other types of symbols may also be useji. TTie types of rncasurement 
we will consider arc numbering, cOunliog, ranking, and measurement 
in the '‘restricted" sense. As already indicated, the latter refers to 
procedures which makes use of a constant unit of measurement; it is 
to this type of procedure that most people refer when Uiey use the 
tenn measurement. However, we are using the term in a much bro-ider 
sense. 

Tlie numbering of classes of objects or events involves wliat ha? 
cemve to bn known ns a nominal scale. Ranking involves what is 
known ns an ordinai scale. Measurement m the restricted sense in- 
volves either an interval or a rolio scale. We shall consider each 
of these in turn. 
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NUMBERING 

In its simplest fonn numbering is Ihe process of assigning to ob* 
jccts or events numbers which ecni'e as eigm or munei of (i.c-» 
nale) these entities. For example, college students, baseball players, 
and automobile registrations are assigned numbers for identification 
purposes. Such a process makes it possible to use uniform and shor 
“names" and facilitates filing and cataloguing information about these 
entities. In order to number a set of entities they must be distinguis i* 
able from each other and they must retain their identity (distinguis 
ing properties) over the period during wliicii the numbers are to e 
used ns names for llicm. 

It is also possible to employ numbers to designate properties. For 
example, we may have a list of properties to which numbers are as- 
signed: 1 may represent red, 2 may represent yellow, and so forth. 

We may, of course, combine these two types of numbering. Part ® 
the number assigned to an entity may designate the entity, and pa 
may represent one or more of ita properties. For example, in tne 
assignment of Dewey decimal numbers to books in a library, part o 
the number designates the book and part designates its subject ma 
ter, Tliat is, part of the number designates the class of books to 
which a particular book belongs. . 

Since numbers arc generally assigned serially, they can also be us 
to represent some ordering property of the entities. For exampf, 
bouse numbers indicate Ibc approximate location of the bouse on the 
street, and army serial numbers tlic approximate time of induction or 
enlistment of the person involved. 

It is important to observe that in the types of numbering we ha^e 
considered no arittimetical operations can be performed on tlie numbew 
to draw any inferences about the cntUica or properties involved. ^ 
fact that we can use letters of the alphabet or other symbol sequences 
in place of numbering makes apparent this inability to draw arith- 
metic inferences. 

Nominal Scales 

As indicated above, not only may wc number individual entities but 
we may also number classes of entities. Classifying entities and num- 
bering them involve what is called a nominal (i.e., naming) scale. 

Many types of classificatory schemes can be devised, but the typ® 
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tor tbo number rcprcscnlmg » particular property. in 

two-digit number, I in the Cret position may represent ™ 

the second position may represent "did not graduate from g 

“to”?entities have been classified we can count Ibe number ol Ibcm 
in c“rcts and compare tlio resulting dislribulion ol " 

classes at different limes or the distnbutioo ol different 
subjected to the same classification. Allliougli no cannot P"'“™ “ 
arithmetical operations on the class numhem, ^.1.“ 

sidcr the statistic called the mode ol the distribution ol the entdics. 
The mode class is the one nitli the largest number ol members in 11. 


Counting 

Counling, in its simplest form, consUta of matching cod5CCuU%c 
posiUve mtegers to elements ol a class. (This is equivalent to assign- 
ing “ 1 ” to each class memher and adding tlic Ts which hate 
assigned.) The largest numher used is then Uic number ot elemen 
the class. In some cases the elements (units) may be divisible inw 
parts, in which case counting becomes more complicated^ . 

assign appropriate fractions to parts of elements and 1 ® . 

whole unit and add the numbers assigned. Such is Oie procedure 
counting money where the dollar is used as a unit. Determining 
appropriate fraction to assign to parte of units may inTolve 
ment in the sense to be discussed; for example, in counting the n 
her of rolls of paper on a shelf where one or more rolls have ee 
partially used. , 

Although counting is a relatively simple concept, it may mvo 
very complicated operations. The complexity may derive from ^ ^ 
sources; Hp finin g elements and matching numbers and elements, 
example, in counting the number of people who live in a hou^» • 
may not be clear who is to be considered as a resident. Shorn 
count a son away at college for eight months out of the year or 
maid who sleeps in five nights and spends the others with her faim y 
at their home? It is essential to have good definitions of the uni o 
be counted. . 

Counting the people in a crowded room may be difficult even t oug 
we have no difficulty in identifying a person. This may be due to 
fact that they will mill about. To count them w e may have to ^ 
them stand still or put them in line or have them pass through a oot 
When we deal with units that nun’e during the period require o 
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count thorn, it is necessary to find a procedure for counting each one 
once and only once; otherwibo we ciUtcr underenumerate or onerenu- 
merate. Considerable ingenuity may be required to find or e%'en ap- 
proximate such a procedure where human beings are involved: wit- 
ness the taking of a national census. Difficulties may arise even where 
inanimate tilings arc involved, as, for example, in taking an inventory 
of material in a factory or goods in a depArtment store which cannot 
be shut clann during the process. 

The operation of comiting is similar to numbering in that numbers 
and entities arc matchcl. but, in counting, tfiis is not done in order to 
“name" the entities. Tlie objective of counting is to obtain a number 
which represents a property of the class of entities which are involved. 
The property of Uic class, of course, is the numher of unlU it contains. 

Ail arithmetical operations arc applicable to the numbers obtained 
by eousting. 

Probability Measures 

li will bo recalled from the earlier discussion of functional prop- 
erties that Uicir measurement always involves the determination of 
probabilities: either the probability of production (in the form of an 
individual producing a course of action, of probability of choice, or of 
a coun'C of action producing an outcome) or some function of sueli a 
probability (c.g., the rate of change of probability of choice with 
respect to time). II will further be recalled that the determination 
of what were called ob;cctU'« probabilitica involves the estimate of 
the limiting relative frequency of a subset of events. The determi- 
nation of relative frequencies involves two counts: (1) counting the 
number of instances in which events in a specified subset occur, and 
(2) counting tlie number of instances of events in a specified set 
which contains tJio subset. Consequently, objective probability de- 
terminations arc reducible to Ujc ratio of two counts. 

Ail aritlimetic operations can be perfonned on probability "meas- 
ures," but their meaning (i.e., operational significance) is not neces- 
sarily obvious. For example, the sum of the probabilities of two 
independent (i.e., not causally connected) events occurring (Pi -|- P,) 
is not tlie jirobability that both events will occur simultaneously, but 
tho probability that one or Ujc other or both will occur. The product 
of these probabilities, PjPj, is the probability that both events will 
occur rimultoneously. The calculus of probabilities specifies the 
significance of each aritl»fl;ctical opeialion on probability measures. 
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RANKING (ORDINAL SCALES) 

Ranking involves ordering the elements of a set with respect to some 
(dyadic) relation or relations defined over the elements. Thus rank- 
ing is a way of relating some or all of the elements in a set. To under- 
stand the various kinds of ranlung it is necessary to understand some 
of the elements of the logic of rations. To facilitate our discussion 
of relations, we will conuder n set, S, of objects and let x,y, 2 , * " 
be variables ranging over 5. That is, the values of these variables are 
the members of S. Wc let R denote a relationship; thus, xRy means 
that X bears the relation iZ to y. R' is the complement of R; that iSi 
xR'y is true if and only if iRy is false. Thus, if 1? is the relation >> 
then B' is ^ 

Types of Relations 

Tlie four critical properties of relations that are significant from 
tlio point of view of measurement aro refiexiin'ty, symmetry, transi- 
tivity, and connectivity. 

Before defining these concepts precisely it may be helpful to do 
loosely. A relationship is re^extve if it holds between a thing and 
itself; for example, the relationship "is equal to” as applied to num- 
bers. A relationship is symmetric if, when it relates ono thing to a 
second, it similarly relates the second to the first; for example, “is 
married to” in a monogamous society. A relationship is transitive if, 
when it holds between one thing and a second and between the sec- 
ond and a third, it also holds between the first and the third; for 
example, among the integers. Finally, a relationship is con- 
nected over a set of elements if it applies to every pair of these ele- 
ments t.'ikcQ in one order or another; for example, “>” over the set 
of positive integers. Kow we turn to more precise definitions. (See 
Table 6.1 for a summary of these definitions.) ■ 

(la) R is reflexive (in 5) if and only if, for all r, zRx. E-xamples 
of reflexive rclaUons are (in the set of real numbers), “resem- 
bles” (in the set of human beings), and "is similar to” (in the set of 
closed plane figures). 

(16) R U urejfertt'e (in S) if and only if, for all x, xR'x. Examples 
of irreflexive relations arc “<” (in the set of real numbers) , “is 
ried to" (in the set of human beings), and "causes” (in the set of 
physical events). 
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(Ic) R is nonrc/lcjrnt <tn S) if and oaly if there is an or such that 
xRx, and llirre is an x such tliat x/f'j. Exauijdea of nonreflexivo re- 
lations are “employs" tm the set of huuiim beitigsl unci "equals the 
square of” (in Uio set of leal numbers). 

(2ii) ii is 5.UJ»mr<ru- (in Xl if an«f only if for all x and y (not ncecs- 
sarily ilisliiirt), xUy J) yltx' I-lsanipIcs of symujctric relations arc 
(in the set of r-uI numtit-r-i, ■'u niarrictl to” (lu Uic set of iiu- 
luun beings), and “u parullel to" (m the set of lines in tlie Euclidean 
plane). 

(2h> U is asymmilrie (in S) if <uid only if for all x and y (not neees- 
.sarily distinct), x/fy D ylt'x. Examples of asymmetric relations arc 
(in tile domain of real numbers), “is a parent of” (in the set of 
human beings), and "Ims greater mass Uian" (in the set of pliysicat 
ob;ccts}. 

(2c) Ii u noneynimctnc (in S) if and only if there is a couple, x 
and 1/ (not necessarily distinct), such that xRy and ylix; and there 
U a couple, X and y (not necessarily distinct), surh tliat xRy and 
yR'z. Examples of aonsymmctric refatioos are "is less than twice as 
largo as” (in the set of real numbers), “is a brother of” (Jo Uie set of 
human beiugs), and “is a subset of” (m the set of seta). 

In Uio study of any set of objects, an important coosideratioo is 
the criterion of idenlily. That is, ue must ho ahtc to determine the 
conditions under whicb x and y arc to be regarded as the same object. 
The IdcnUty relationship is u^ in the assertion Uiat Uicy are the 
smne. We shall use I to denote the identity relation. Thus xly 
means that x and y arc identical, and x/'y means that x and y uic not 
identical; tiiat is, they are distinct. 

(2d) B is anlUymmetnc if and only if, for all x and y (not neces- 
sarily distinct), \zRy and yRx\ D xly. Examples of autisj-mnielric 
relations arc and "is Uie square of" (in the set of real numbers). 

(3a) R is IransUife it and oaly it, for ever}’ x, y, and z (not neces- 
sarily distinct), \xRy and y/lxj 3 xKs. Examples of transitive re- 
lations arc (in the set of real numbers), "is an ancestor of” (in 
the set of human Iwinga), and "is north of (in the set of geographical 
locations). 

(3h) R is inlyiitmUi-6 if and only if, for every x, y, and z (not 
necessarily dialinct), {xRy and yRz\ 3 xB'i, Examples of intransi- 
tive relations are "is the fatlicr of" and "is married to" (in the set of 
monogamous human beings), “immediately succeeds” (in the set of 

‘The R’mbol “3” meixiia "implies.’’ Tti&t is, if P &ad Q sre statements, "P 
D 0" ii true jl P is" M '■II f* is *nic, Uien 0 is true." 
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integers), and “is perpendicular to” (in the set ol lines in a Euclidean 
plane). 

(3c) 12 is nonlransiLive if and only if Uicro is a triplet, x, y, and 
2 (not necessarily distinct), tucli Hint xlly and yUz and xl2r; end 
there is a triplet, x, y, and z (not necessarily distinct), such that xRy 
and yl2z and xli'z. Examples of nontransitivc relations arc "is dif- 
ferent fronj” (in any set), "is a tdoad relative of” (in the set of hu- 
man beings), and “has a nonnull intersection with” (in the set of sets). 

(3d) R is gwasi-lransitiue if and only if for every triplet, x, y, and z 
(not necessarily distinct), (ilfi/ and ijRz and xt'z\ D xllz. Examples 
of quasi-transitive relations arc “is a broUjer of" (in the set of human 
beings), “is parallel to” (in the set of lines in the Euclidean plane), 
and “is different from" (in any set). 

Not all possible combinations of these properties arc logically con- 
sistent; that is, some involve contradictions: For example, a re- 
lationship which is asymmetrical and transitive cannot be reflexive. 
If a relationship is symmetrica) and intransitive, It must be irrcflcxivc. 
If a relationship is transitive and irrcflcxivc, it is osymmelrical. A 
relationship cannot be asj'nunctrical and reflexive.* 

Reflexivity Is a property of a relation which is useful primarily la 
establishing similarities, whereas irreflexivity establishes differences. 
This 18 apparent from comparing "=” and“>". If a given relation is 
symmetric, then all members of pairs bearing this relation to one an- 
other may be substituted for each oUicr in an appropriate context 
For example, thbgs which “look alike” or "are Uio same size as” may 
replace each other in appropriate contexts. 

Aej’mmetry allows us to dislinguisl) or individuate elements, as m 
“does not look like” and "is a different size from.” 

Transitivity allows us to join tiling together in a chain, m a 
hierarchical or ordered series; /or example, “>” in the set of integers. 

(la) R is connected if and only if, for every z and y, xVy D [xf^y 
and/or yRx]. Examples of connected relations are (in the set of 
real numbers), "is older than” (ia Uie set consisting of a given person 
together with his father, paternal grandfather, paternal great-grand- 
father, etc.), and “has a la^r radius than” (in the set consisting of 
all circles with a common center). 

(4b) R is nonconnected if and only if there is an x and y ixl'y) 
such that xRy and/or yRx, and there is an * and y (x/'y) such that 
xR'y and yR'x; lor example, "is the father of" (in the set of people) 

* reader ahouki try to rerily thesa asaertiocs, 
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Property 


Table 6.1. Tm» or Rzlatioxs 
Defiiutioa 

(ndative to a Set of Elements, S) 


Esamples 
(S =■ The Set of 
Positive Integers) 


lo. Rellejavo arsrRi 

I6. Irreflexive x:xR'x 

Ic. Nonrcflexrte I3r: |3*:*Rx| 


2a. Sjinmotric 
26. Asjiniiictno 
2c. Nonsymmetric 

2>!. Antisj-nunclric 

3n. Tranalivo 
36. IntransiUvo 
3c. NontrtoslUvo 


3J. Quui-traositlve 


X, 1/; xUa syRt 
I, y;xR?3y/l'* 

13 X, y: xRv *"<1 
[3x. y.zRs and yB'il 
X. 

X, y,t: \xRyiodyRi\oxRt 
X, y, x: \xRy sod yR^\ ^ 

3*. y. *: \i^V< 

and 

3i. V. *'■ 1'^’ “** 

x,y,t- lilly. yRx, eod xi'x) 3 
xRt 


4a. Connected 
46. Konconnccled 


X, y. xl'v o nnd/ot y/Jj) 

a’x, y(*/'y)* “‘IM 

3 j, vixl'y)- 


Equals the square of 


Is less than t^ice 
As large as 


Immediately succeeds 
Is less than t«)ce as 
largo aa 

la difforeni from 

> 

Immediately succeeds 


Read "x:” as "for every 
as ''implies.” 


*• 3x:” as '‘thcro exists an i such that,” and "=» " 


U . relation i. .onntclcJ, 
of a population. 


Types of Ordering 

The elemenU of a Th^ orclcriogs 

Ll:r» "y »' “! 

S that 0.0 
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tliat is cither asymmetric, oonsymmctric, or antisj-mmctric. A rela- 
tion such as ‘‘=," tberc/orc, cannot be used to order clcnicnta because 
it is symmetric. A relation which is either nonsymmetric or nsj'm- 
raetrio and which docs not hold between all elements ol the popu- 
lation (i.e., is noncoonccted) provides the weakest type at ordering. 

The ordering is strengthened if the relation holds between all pairs 
of different elements of Uie population; that is, if the relation is con- 
nected. For example, provides such an ordering for real num- 
bers, and “costs at least as much as” for items which arc for sale. 

A relationship which is transitive and symmetric (e.g., “ = ”) allows 
U3 to form chains of similar things, but not to order them. To obtain 
an ordered chain we require that Uio relation be either asymmetric 
(eg-. ">”) or anlLsymmetric (c.g., 

A partial ordering or ranking is formed by a relationship which is 
rcflexii’e, antisymmetric, transiUr'c, and nonconnected. For example, 
yields such an ordering of automobile license plates. Some plates 
which have only letters and no numbers arc not comparable with Uiose 
liaving only numbers. Thus n-c get a partial ordering of the popu- 
lation. Partial ordering reficcU the mulUdimcnsicnality of the attri- 
butes di^erentiating the elements in tlie relevant set. 

A weak ordering or rankiag is formed by a reflexive, aotisymmelnc, 
transitive, and connected relation. For example, yields such a 
scale of people’s weight. Since some people have the same weighti 
they are not ordered with respect to each other and hence the weakness 
of tlie ordering. Another weak ordering is provided by military ranks. 
Each rank constitutes an equivalence class, and for any two officers 
(x, y) cither x&y, or yStx, or both. 

A complete ordering or ranking is formed by an Irrcflcxive, assj’Oi- 
metric, transitive, and connected relation. For example, “>” yields 
such a scale of the positive integers. Such an ordering is also called 
a chain, a simple order, or a Imeor order. It yields a complete rank- 
ing of every element of the population. 

There are many variations of ordering. One of tlie more sophisti- 
cated procedures, developed by Siegel (1956), has been discussed in 
Chapter 3. This scale yields a weak ordering of both utilities and 
the differences between them. 

Numbers are usually assigned to ordered elements to represent their 
rank. In the case of a partial ordering this has little value, since 
some of the elements cannot be ranked In a weak ordenng the same 
number must be assigned to elements which have the same value of the 
property involved. In a complete ordering each element can be as- 
signed a number of its own. It is conventional to use “I” to designate 
either the lowest or highest poation in the order and to number serially 
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Table C.2. PBOPEBTiEa op Tttes op Oisbejuxo 


Property 

Partial 

Weak 

Complete 

Ildlexive 

Irrefladro 

X 

X 

X 

Asymmetric 

Antisymmetric 

X 

X 

X 

Ti-arisiU%-e 

X 

X 

X 

Connected 

Naocoouectei 

X 

X 

X 


tJjercafter. Aritbrnetica! operationa cazioot be per/ormcd on tbc niuu- 
bcrs 50 assigned. For example, if (Itrce objecta arc ranked by n ciglit, 
it does aot follcfe that the n-cight of tlic heaviest (3) is equal to tlio 
sum of tho Keiglits of the lighter tuo (1 + 2). Hoe is the welgliii of 
"3" necessarily equal to tlirce times tl»e weight of “f." Tliis Is made 
apparent by the fact tii.al letters (A, If, C, ■'■) can also be used to 
designate rank nithouc any loss of information. 

Table 0.2 suonuartzes the basic properties of partial, weak, and 
complete ordering. For further discussion of other aspects of oriinal 
scales see Coombs (lOol), Coombs, ItalfTa, and Thrall (1954), and 
iitevens (1951 and IQ50). 


AfEASUKEMENT (LV THE RESTIUCTED SENSE) 

Afcasurcmcat in tJie restricted bcnsc iDTOh*cs tho use of a constant 
unit of measurement. Such a unit is ciuploj-ed in both interval unci 
ratio scales, 

Interval Scales 

Interval scales correspond more closely tlmn do nominal and or- 
dinal scales with what we normally think of as measurement. 
Whereas neither numbering, counting, nor ranking explicitly in- 
volves a constant unit of meosarmenf, thfl interval scale doc.*?. But, 
03 we shall sec, the unit of measurement imotved in nn intcr^'al scale 
is arbitrary. 

Tho essenUal characteristic of the intm-.al rcalc is best undcretooil 
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by considering one property of the onlintil scale. Consider Uirce 
persons. A, B, and C, such 0»at we have determined that A is taller 
than B and U is taller tlian C. Then we can rank them as 1,2, and 3. 
In the usual ranking procedure we do not know wlicthcr A is as much 
taller than B as B is than C; Uial is, we do not know the relationship 
between A’s height minus li'a and B*s hciglit minus C’s. In tlie higher- 
ordered metric scale dc\cloj>ed by Siegel (1950), these differences arc 
ranked, but tlic ratio of their magnitudes is not determined. The 
measures obtained from an interval sente enable us to m.akc assertions 
about, and draw iulcienccs from, Uic magnitudes of the dUTcrenccs 
between tiie properties of different objects or events. 

Tim centigrade and Fahrenheit scales of temperature arc intm’al 
scales. Equal intcivala of temperature arc sealed off by noting equal 
volumes of expansion of tlic liquid used in Uic tlicrmomctcr. 

These scales have the important property of being “unique up to a 
linear transformation.” The meaning of Uiis property can be ex- 
plained as follows. Suppose that ue obtain measuremenU of a set of 
elements {rtj. These can be transformed into otiicr values (r/) by 
the linear equation, xf « oxi + b, where a 5 * 0. The relative magni- 
tudes of the new values arc the same as U»cy were originally. Graph- 
ically, this is illustrated in Figure 6.1. 

These scales do not have any “Dalural” zero point. The aero point 
is usually set arbitrarily or tor convenience and is then accepted by 
convention. Hence, the centigrade and Fahrenheit scales have dif- 
ferent zero points. These scales can be transformed into each other 
by a simple linear transformation: 
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where tf is Uie temperature on tiro Fahrenheit scale, and f, is the 
temperature on the centigrade scale. 

It can be misleading to say that my valve on an interval scale is 
some multiple oj aiiotlier; for example, Uint an object at 6-1® F has 
twice Uic temperature of an object at 32® F. That tliis is misleading 
is apparent by boUi example and general considerations. If ue trails- 
fonn 32® F and 01* F to tlic centigrade scale, wo obtain 0® C and 
17.8* C. Tlic latter .ire obviously not in a 1 :2 ratio. More generally, 
even if Ja = 2rj, Uicn, unless 6 = 0, 

0X2 + bpi 2(ttrj 4- fc) 
since a(2xi) + b 2(ari + h) 

'iox, +h 9^ 2aii + 26 . 

These will only he crjual in the special case where 6 = 0. On the 
other hanci, wc can express the di/fcrcncxt (intervals) between values 
on the inter^'al 6c.ile as multiples of each other 6i®F is twice as 
far from 0® P as is 32* F). This can he seen hy comparing the inter- 
vals hetween tlircc points, xi, Xa, and xs. The ratio of the interval 
(xi - Xi) to the interval (Xj — x») is, of course, 

xi -xs 

Xi — X} 

If wc transform the values of x,, Sz, and x,, Uus value of the ratio is re- 
tained: 


(axi + b) — (axz + 6) axi — axz Xi — Xz 
(oxj + h) — (0x3 + 6) oxa — 0x3 I* — xj 

The properties measured on an interval scalo can be mapped into 
tlic real numbers. AUhougb some arithmetical operations cannot be 
performed on these numbers (e g., addition and multiplication) with- 
out special explanation, most arilhmcUo operations can be perfonned 
on tho differences (intervals) between all pairs of these numbers. 

The type of interval scale we have considered here is linear, but 
the intciwal involved need not be linear. Stevens (1939, pp. 31-34) 
discusses another type, the logarithmic Interval scale, in which tlic 
transformation is 
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Although use for such a scale has not yet been found, it is of theo- 
retical interest and opens Uic possibility of development of a wide 
variety of intcn’al scales. 


Ratio Seales 

The cliaracteristics of this tyjMS of tcalc hare been summarued by 
Stevens (1951, p. 8) as follows: 

... Its numerical values can be tnmsfoni»e«l (as frotn inches to feet), only 
by multiplying each value by a constant, ilut is, ii =» otj. An absolute 
zero is always implied, even though the zero value on scales (c g., Absolute 
Temperature) nuay never be produced. All tyi>cs of statistical measure are 
applicable to ratio ecaic.*, and only with these scales may «»o properly h>* 
dulge in logarilhmio transformations. . . . 

Measurement on a ratio scale is achieved with a constant but ar- 
bitrary unit (standard) of measurement (c.g,, feet or meters in meas- 
uring lengUi). 

Any one value on a ratio scale can be oepressed as a multiple of 
any other. For example, "3 feet” is equal to “3(1 foot),” and this 
relationship remains invariant with transformations of the type 
= as; that is, “30 inches” is 3(12 bches). Or we may say that one 
body is twice as dense as another if the ratio of its mass to its volume 
is twice tliat of the other body 

All aritlimetical operations at« applicable to the (real) numbers 
obtained from measurements on a ratio scale. But some of these 
operations — for example, addition — may not have the apparent opera' 
tional significance. For example, the addition of the densities of two 
bodies, D\ and D-, is not equal to tire density of the two bodies taken 
“jointly.” That is, Dj = itfi/F, and C* = i'fa/l's (where M and V 
represent mass and volume, respectively), but the density of the two 
bodies taken jointly is not equal to Di Dj or ilfj/Fi + Ms/Vs, ‘t 
is equal to (.l/i + Afs)/{Vt + V,). (We shall consider the measure- 
ment of density in more detail below.) 


SUMMARY OF SCALE CHARACTEIUSTICS 

The characteristics of the four types of scale which we have con- 
sidered have been succinctly summarized by Stevens and are shown rn 
Table 6.3. Stevens has also summarised the statistical measures which 
are appropriate to measurements made on each type of scale. These 
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Table 0 3. A CussmcAXioN of Scales of MEi3VHLiiF.ST * 
ri:i$ic Empirical AlaUtematiuil 

Scale Oiwrations Gruup Structure Tj pleat Exainple.s 


Nosji.val 


Ordinal 


I.STEKYAL 


Ratio 


Detenniniiljwi 
of eqimlity 


Detemiuiattun 
of greater or 
less 


Determination 
of the eriuolity 
of iutervuU or 
of ilifTercnccs 


^Vnuulation group 

= m. 

where yXz) means 
any one-to-one 
suUsUtulion 
I&otonir group 

r-fix). 
ivbere/(je) means 
any increasing 
monotooio 
function 
Liiieer or afline 
group 

j' m ax + 1, 
a>D 


Detem^oation Similarity group 
of tlio equality x' — ex 
of ratios 

OO 


“Numbottng" o! football 
players 

Assignment of type or 
motlel numbers to classes 


ffuitlness of minerals 
Street nuiabers 
Grades of leather, 
lumber, nooJ, etc. 
Intelligcivce-tcst raw 
scores 

Temptraluro (Fahrenheit 
or Celsius} 
rosition 
Time (calendar} 

Eoergy (potential) 
IntcUigcDcc-lesi "standard 
scores" (7) 

Numeroslty 

Length, density, work, time 
inteni'als, etc. 
Temperature (ftankine or 
K^ein) 

Loudne!>s (aoncs) 

BrigUtne.ss (brils) 


• (FriDW Stevens <1039, p. 25). . . . The basic operatiofjs needed to create a 
given scale are all those listed in the second column, down to and including the 
ojicratiim listed op/xidte the scale. Tlie third colunut gives the mathematical 
tranaformationa that leave the scale form invariant. Any numeral x on a scale 
can be replaced by anoUier numeral x', wlrerc *' Is the functiem of x listed in 
column 3, 
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are sliovrn in TabJe G 4. It should bo noted that the scientific meas- 
ures applicable to any type 0 / scale include tljose listed for the other 
types of scale located above it to the table,' that is, tljcsc properties 
are cumulative. 

There is notiiing in the nature of the four types of scale which ha%c 
been discussed to assure their exclusiveness or exliaxxstivcness. 
Coombs, Kaiffa, and Thrall (lOoi) Itavc already shorvn that addi- 
tional scales can be generated by 'Tanning mixtures.” We can ex- 
pect the construction of new types of scales in the future and, per- 
haps, a more effective classification of possible types of scales than 
is currently available. 

In examining the scales which have been discussed, one can hardly 
avoid the impression that they form a hierarchy of some kind. In one 
sense the nominal scale seems to be the most elementary and simplest 
type, and the ratio scale the roost complex. But despite the complex- 
ity of the raUo scale it also appear* to be more fun^rocntal in some 
sense; at least it has struck some measurement theorists in this way, 
particularly N. R. Campbell, whoso views we examine laUr. 


Tabix 6.4, ExAairi.es or Statisticai. Meascbu Atpsopeutb to 
hIcASenesanm SIaoc ox the Vabiocs Classes op Scales * 


Scale 

Measures of 
Location 

Dbpcrsioo 

Association or 
Correlation 

Significance 

Tists 

XoitrsAL 

^fode 

Information, 

B 

Infonnation 
transmitted, T 
Contingency 
correlation 

Chi square 

Oedisal 

Median 

Percentiles 

Rank-order 

correlation 

Sign test 
Run test 

IsTEBVAL 

Arithmetic 

Stacdaid 

demlion 

dectalicoi 

Product-moment 

correlation 

Correlation 

itest 

Ftest 

Ratio 

Geometric 

Harmonic 

Percent 

variafion 




From Stev'cns (19S9, p, 27). 
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TIIE NATURE OF ^^EASUREMENT 

There is anything but common agreement among scientists and 
philosophers of science as to just what measurement is and how it 
should be performed. These points of view can be characterized as 
ranging from iJie very restricted to tlie s'ery general. By reference 
to the worii of N. R. Campbell, S. S. Stereos, and C. IV. Cliurchman 
wc shall try to present tlie more signiGcant points of Mew and to 
identify tho important methodological problems involved in measure- 
ment. 

Campbell’s Concept of runJanienlal .Measurement 

Campbell, operating in tlie spint of otiicra wc have considered who 
have sought simple or irreducible concepts or operations, was pre- 
occupied with what lie called JuniJumental McotuTfment, la order 
to determine whether a property is subject to fundamental measure- 
ment, Campbell has set down sL\ conditions on tJie way that tblngs 
having Uie property can be related. The relationsliip must be: 

(1) connected; lor evety pair of elements, s and y, cither xRy, j/Rx, 
or X o y; 

(2) atymmetric; zJiy D yii'z; and 

(Sj transitive: (sRy and yUi) xBz. 

The relations “is .it least n» lieaty os” and ‘hs at least as dense as" 
satisfy these requirements for the class of "physical bodies." But 
"is at least as red as" and "is at least as dark as” arc not connected 
relations. "Some colours are neiUicr redder nor less red than others, 
two shades of tho same blue, for example; and yet they are not ‘equal 
in colour' . . [Campbell (1957, p. 272]. Ilcnce, according to 
Campbell, we must drop color from the class of fundamentally meas- 
urable properties, at least with respect to Um set of physical bodies. 

Ne.tt, a fundamcnUl ijic.'isurtaiicnt, according to Campbell, has tho 
following properties of additivity: 

If a and fl represent tlie numbers assigned to the dements repre- 
tenling the property involved, then 

(•i) 

(5) = 

(6) {« + /!) >«, where /J>a 
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The measurement ol wei^t by means of a balance, according to 
Campbell (1937, p 279), satisfies these conditions; 

We state that the weights of two bodies A and B are "equal” wh^, if ^ 
is placed in one pan of the balance and B in the other, the final position of 
the pointer of the balance is unchanged. We say that the body C is "added 
to” the body A, when A and C are placed in the same pan of the balance; 
and that it is "subtracted from” the body composed of A and (7 in the same 
pan by removing C from the pan. When we have thus defined “equal” uid 
"added to” in the use of the balance, we can state, corresponding to arith- 
metical proportions which involi'e addition and equality, propositions about 
what will happen to the balance when wc place bodies in the pans. 


It is clear then that weight satisfies conditions (4), (5), and (6). 
Density, even though measured on a ratio scale, according to Camp* 
bell, does not satisfy condiUon (6). iVs already noted, if we “join” 
two bodies of density a and the density of the conjunction is not 
a -f whereas, if a and Q represent weights, this would be true. 
Weight then is a fwidamenial magnitude, but density is derived. 
“It presupposes the measure of mass or volume” [Campbell (1957, 
p. 277}]. But weight "apparently" involves no other measures and 
hence is fundamental. 

The apparent diScrence between scales of density and of weight with 
respect to additivity also appears to exist between scales of “return 
on investment" and “profit.” Return on investment b usually meas- 
ured as a percentage: 


(100) 


net profit in $ 
total amount invested in S 


If a corporation has two companies, each obtaining a 5 per cent re- 
turn on its investment each year, Ibe total return on investment is 
5 per cent, not 10 per cent. On the other hand, if each company makes 
Sl.000,000 profit per year, the total profit is $2,000,000. Profit satisfies 
all six conditions set down by Campbell but certainly is not a funda- 
mental measure in the sense of depending on no other measures; it 
depends at least on a measure of expenses and income. On this point 
we shall have more to say below. 


Critiques of Campbell 

Stevens, who was so concerned with allowing only definitions con- 
taining operations wbidj could beraluced to a fundamental operation 
(pointing) , is not so ardently a fundamentalist when it comes to meas- 
urement. He was very disturbed by the Committee of the British 
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A^ociation for the AdvAnccmcnt of Sdencc, which issued a report 
oa the problem ol measurement io 1940. This report, dominated by 
the participating Campbell’s point of view, attacked one of Stevens’ 
sensory scales. One member asserted that “any law purporting to 
express a quantitative relation between sensation intensity and stim- 
ulus intensity is not merely false but is in fact meaningless until a 
meaning can bo given to the concept of addition as niiplicd to fecn-ia- 
tion” (Ferguson (1940, p. 245)1. 

Stevens (1949, p. 677) replied as follons' 

Paraphrasing N. R. Campbell (FwuJ Report, p. 340) wc may siy Uui 
(iir.’ifurcBicnt, ia the broadest sense, <s dc/ha^ as the assignment of numerals 
to objects or events .according to roles. Tbc fact that numerals can be as- 
signed under different rules leads io different Unds of scales and different 
kinds of measurement. The ptoblon thco bc-cooics that of making explicit 
(o) the vanous rules (or the ssregnment of nuuierxCs, (b) the mathematieal 
properties (or group stroctutc) of the resulting scale*, ami (c) the gtatUlic.il 
oiwratioM applicable to mcaruremcot* msde trith each type of scale. 

Stevens docs not object to Uic concept of fimdamcntnl uicasurcmcnt 
—in fact, ho embraces it — but he docs object to restricting the con- 
cept o( tQcasvitctucnt to (undamental mcasuTts or ones dcrivetl from 
thenn To support his poaition be defined four types of scales which 
can bo used in measurement (in the broad sense of the word) : nominal, 
ordinal, inten-af, and ratio. 'Hicsc types have become the focus of 
most subsequent discussions of mc.'isurcmeflt, Including this one. 
Stevens would like ultimately to J»ave all inc.asuro3 derived from onw 
tliat arc fundaiaental in C.ainpbcJrs sense, but io Ujo meantime be 
argues that the “weaker" forms of number assignment arc also meas- 
urement. 

Churchman (1919, pp. 4S3-^S5) made a much more “Xundamentar 
attack on Uic concept of a fiindanicnlal scale: 
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say the operation makes A smalier than J? (A < B), then there roust not be 
any prescribed operation leading to B<A. 

Now it seems to Campbell that lengths, areas, weights, etc., are additive 
magnitudes, for they are all based on the choice of a certain standard mag- 
nitude; the operations consist of taking parts or multiples of the standard. 

But none of this substantiates the claim that the measuremcots of 
length, weight, etc., arc fundamcata] in the sense that they do not depend 
on the measurement of any other magnitudes. The natural question we 
should like to ask someone ndghing objects, is whether the arms of the bal- 
ance are equal and convection currents absent. And, in general, we irant to 
know what variables must be checked to make sure the balance is behaving 
properly. But to know the answer to ibcsc questions it certainly looks as 
though the experimenter must take incaEuremenls of other magnitudes, and 
hence the detenniaation of ircight is not “fundamental’' in Campbell’s sense. 
No, says Campbell, all the experimenter need do is follow through the oper- 
ations and “discover whether inlcrcbange of the contents of the pans affects 
the balance.” As a matter of fact, Campl^ll thinks the question about all 
tlie necessary checks on the babnee b “actually dangwous.’' “For the an- 
swer must involve in some form the theory of the babnee. ... We are bas- 
ing our measurements on theory, rather than our theory on measurement, 
wWch is a most dangerous procure.” Dangerous for whom? Campbw 
does not tell us, but we are led to suppose that the danger is for any good- 
beatted fundamentalist anadoua to get bis start somewhere. It is interesting 
to contrast Brown’s dictum: “The discovery of the law makes the posabihly 
of exact and accurate measurement rather than the measurement makes 
possible the discovery of the law.” To refute tha position, Campbell argues 
that to check the process of weighing, we "amply try out" the objects on 
the pans to see if the operations satiuy llie for^ conditions of additive 
magtdludes. Well, bow many objects must we try? W’ho shall try them? 
On what days? On the critical question of judgbg a result, Campbell as- 
serts that the detemunation of lengths, for example, "depends on judgment 
ol the contingmty of parts of lines, which is a relation fnetinctiuely Pfr- 
caved." Do all people instinctively perceive alike? Does one person in- 
stinctively perceive ahke at diOerent limcsT Of course, I don't know exacuy 
what Campbell means by "instisctiirly peredved"; but almost any conceiv- 
able meaning of the term would lead to a decided negative to the questions. 
Different observcri do pcicove “insUnctivdy’' in different ways, even when 
the operations arc very carefully specified. Further, different observers get 
signifiMnUy different results on different days, in different bboratories, etc 
I think Campbell’s position might be tolerat^, even in the bght of its bek 
of rigor, if scientific experience indicated that the results of followmg oper- 
ations at different times, by different observers, dtd not differ very widely 

Measures always presuppose and hence involve other measures 
Control of the environmental conditions, which is always necessary 
some degree in measurement, itself always requires ineasuremiid 
Therefore, measurement, like deGning, has tlic appearance of 
larity, but it actually cycles in "Uiree-dimeDsional space," progressing 
aa does defining. In a specific study, however, it is always desirable 
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to specify a measure in such a way that tlic presupposed measures or 
other ineasui-cs required are less difficult than tlie diicct measure in 
the context of tlio problem or question. One of tlie difficulties, for ex- 
ample, in the specification of an idealized measure of value presenletl 
in Chapter 3, uas that it prcsuptwscs a measure of knowledge which 
is at least as difficult lu obtain as is that of value. But, as in tins 
case, the presupposed mcasun's cannot always be more e.asi[y at- 
taiacd tlian the measure diicclly lovulved. Even here the presupposed 
measures should he made explicit so Uiat efforts to improve any one 
of tlie interrelated measures lake the others into account. Only with 
a self-conscious treatment of tiicse interrelations can measurement 
jirogrcss more evenly o\ cr all tlie seicciccs and in all problem areas. 

Churchman’s critique of Campbell’s concept of fundamental meas- 
urement is based on the latter’s assertion of the existence of measures 
which presuppose no other measure. Another aspect of Campbell's 
doctrine is also open to criticism: Uie concept of Additivity. Accord- 
ing to Campbell (1957, p. 2S1), 

la respect to welghiog a process of addition and a relation of equality can 
be fou&d which are similar ... to the atilbmeucal process of addition and 
the arithmetical process of equality, but a process of addition and a relation 
of eqimlity which are thus sunilsr to the antIunetieaJ processes and relahoa 
cannot be found in respect to detcnninalion of density. 

The fact is that we can find a set of operations which yield additiv- 
ity of densities, at least under rcslriclod conditions. Consider two 
eontaioers of equal volume, one containing gas C?j of density Di and 
the other gas G7 of density Ds- If Gt is added U> Gj in the first coa- 
tniner, Uie density of Uic jcsuHing mixture is equal to Di -f Dj. This 
Buggesta the possibility of eventually finding a set of operations which 
more generally yield additivity of density. This possibility is implicit 
(but not intended) in an observation made by Campbell (1957); 
“Thus we may say Uiat A is tlenser than iJ if a liquid can be found 
such that U will float in it and A will sink” (p. 276) . To this obser- 
vation Campbell attached the following significant footnote (p. 276) : 

I am inclined to think that Uus is historically the ultimate meaning of 
density, that to Archimedes, for example, density was simply the property 
in virtue of which some bodiea floated while others Mnk, and the discovery 
Hat this property was reprcsenleJ by the ratio of the mass (or rather 
weight) to the volume was a later and independent discovery. 

Pleasures of properties, like tlieir definitions, may go tJirough an 
evolutionary process. Campbell's observaUons on densi^ indicate 
that the measuremriDt of density has progressed from an ordinal to 
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a ratio scale. The posMbiiily of such evolution is explicitly recog- 
nized by Stevens (1959, pp. 21-25): 

It 13 an interesUng fact that the measurement of some nuanlilics may ha\e 
ptogrcstcd from scale to scale. We can imagine, for cMimple, that certain 
FjVim na might spcak of temperature only os freezing or not freezing and, 
thereby, place it on a nominal scale. Others might try to express degrees 
of warmer and colder, pcrliaps in terms of some series of natural events, and 
thereby achieve an ordinal scale. Aa we all know, temperature became an 
inteni scale with the dcvelopm^t of themometry, and, after th^o- 
dynamics had used the expansion ratio of gases to extrapolate to zero, it be- 
came a ratio scale. 

We can observe just sucli an evolution compressed into a short tune 
in the measurement of utility (value). Man undoubtedly first classi- 
fied things as either desirable or undesirable. Eventually, with the de- 
velopment of preference tests in psychology, ordinal scales were in- 
volved. The von Ncumann-Morgenstero measure of utility involves 
an interval scale. Kaally, serious efforts are now being made to de- 
velop ratio scales of Utility. See, for example, Stevens (1959, pp- 52- 
61). 

What conclusions can be drawn from the critiques of Campbell’s 
work? Churchman (1949, pp. 48S-492), after his study of Campbell's 
doctrine, suggested five principles of measurement: 

(1) We must stop ibinklog of scieotilic method as proccedl&g from the 
mluiuvely simple dements to more complex ones. 

(2) Additive scales are not any more “fundamental’' than any other tyiw 
of scale. 

(3) All exact measurements take place in an ideabzed environment; the 
results of an actual experiment arc to be regarded as estimates of what would 
happen if ideal conditions could be obtained. 

(4) The scaling of a property of an object provides information as to the 
most dHcient use of that property in any problem situation. 

(5) A scale furrushca the experimenter with a technique whereby be can 
measure the degree of scientific perfection he has attained. 

The significance of these principles will become more apparent as 
we address ourselves to the following questions, which should be an- 
swered in preparation for measurement in research: 

(1) Definition. What proper^ of what objects or events is being 
measured? What operations should be performed on these objects 
or events, and how should numbers bo assigned to the property ob- 
served? 

(2) Formal jiroperties. What mathematical and statistical opera- 
tions can be performed on the measurements obtained? 
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(3) Accuracy. How ean we atijtist mcasurciiicDts obtained in less 
than the ideal conduions that are speciHcd in the relevant definition, 
and what is their accuracy? 

(4) Control. Haw can an acceptable level of accuracy he main- 
tained? 


Definition of What Is Measured 

It may seem unlikely that a procedure can be developed or adopted 
for measuring something without being able to define operationally 
Uiat which is being measured. If this can be done, one can reasonably 
ask, "In what sense can the numbers obtained be called a tneasure- 
ment?" Such a procedure is niucli more common than one would 
expect. Suppose, for e.\aznpJe, that a series of pussies are collccfed and 
given to a number of persona in a specified order. Each person is 
scored on the number of pussies correctly solved within a specified 
time. Then a number of other characteristics of these persons are 
determined, such as average annual earnings, rate of advancement, 
size of family, and JQ. Correlations are then obtained between these 
characteristics nnd the score on the puzzle test. If a high correlation 
is obtained between the test score and any of these characteristics, the 
scorn on the test is used to cstunate the propcrifca with which it is 
correlated. 

Stevens prefers to call such scores as am yielded by this test tiuit- 
contg rather than me-asurcs. Indicants arc effects or correlates re- 
lated to the property in question by unknown laws. According to 
Stevens (1351, pp. 47-4S), toe use of indicants 

... is inevitable in the present stage of our progress, and it is not to be 
counted as a blemish. ■ . . 

The end of the trad is uwasurcmcnl, nliich ve rc.ach vheu we sohe the 
relation between our fortuitous indicants and the proper diaieasions of the 
tiling in question. 

In the meantime we take bold of our pcwdcins by whatever handles na- 
ture provides. ... We measure changes in the resistance of the skin and 
call it an indicant of emotion. . . • 

The distinction between measurK and inJicsnts dissppeare, of couri.o, as 
soon as we learn the quantitative rriation between tlis indicant and the ob- 
ject of our interest, for then the indicant can be calibrated and used to 
measure the pbenomeBon at issue. We measure electric current by menus of 
a cabbiatcd indicant composed of » cmI of wire suspended by a spring in a 
magneUo field. . . . The mote matmie a ecience, the more it uses caMralcJ 
indicants. 
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Recalling that Wioanta arc 

parcat that Ihcit appropriatooaa can Im acea >ra«% ““““ ^ 

it either tl.o causal connection to *“>' l’™P ^ j „ comlation 
esUWished, or Uiat property can bo measured bo t , ^ 

analysis can be pcrlormed. “early, Ibc In™- annoUie^^^ 

the relevant lan-s involved arc unknonn. It lire- ciniilarlv 

then Uie bridBc is intuitive and U by ivbieh their 

in most eases tvliere indicanU are rrsed, Ibc f based on 

use is justified arc not cstablislicd objcctivclj but are a . 

'°to m^y a case aherc indicanta arc iustified ^ 
lation analysis, observations ol the i„a j„ee aill 

Bubjeelive evaluation o! tlie relevant P™P«rt'«' chnreh, 

bo cited beloiv relative to a "test" of rittiludes 
where test scores are correlated with eubjcctne judgni 

Despite these Bhottcominss ot most j^^Ese 

they can be useful as a bridse to measurcmcrit as Ions as tlios ^ 
them do not confuse them rrlth measuring ’“‘".‘"’'"‘'f . bp„.ever, 

done. Even after measurement ot a property is . j.„ 

indicants may be useful because of observationa 
eicampic, tve know how to detcnninc the rigc of a „„ss 

time, but we usually do so by counting Uie nnp i . . t. a- ogg 
section. The rings arc indicanU which arc causally connccta 
but are easier to count than arc the years ol Ufe. 

One geU the impression from the Stcvms quotaUon 
is required to bridge the gap between ind.oanU ^d m 
empirical knowledge. But all tho empirical knowledge m t 
is not sufficient to connect the resistance of the skin smo 
less we have an operational deBnitioa of emotion. Jefinition 
attributes to a lack of knowledge is usually due to a lack ° 

If half the effort which is expended m searching for P^pe 
indicants “quantify” were spent in proper deffning of concep , 
and its application would progress at a considerably grea cr 
In some cases a loose defimtion of a concept is used m 

a measure of it. This always leaves room for doubt as to w ^ 

not the measure actually pertains to the concept involved, 
ample, consider a fairly common procedure for constructing 
tesU, one described in detml by Thurstone and Chave , pg 

They define an attitude as “the ram-total of a mans i 
and feelings, prejudice or bias, preconceived °ot‘ons, i , 
threats, and convictions about any topic” (pp. 6-7). it 
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the "sum-total" is a figure of ^)cccb. Adclitioa in any serious sense 
is not proposed here. In fact, when a dsllnition provides no esplicit 
criterion for relevance of observations or instructions for making them, 
it is not «a operational definition. FurUicrmore, the definition gives 
little indication as to how attitudes can bo measured. 

In the procedure for “measuring" attitudes developed hy Thurstone 
and Chave, statements about tiie topic invohed are selected because 
they seem relevant to tbe topic. Thdr relevance cannot he demon- 
strated objectively. What is nonually done is AS folJoTvs. The state- 
mentg arc submitted to a set oi judges who rank them on tlie basis of 
tlieir own feelings according to tlie sUength of each statement’s favor- 
ahlcnesa (or unfavorablcncss) to Uie topic involved. Those state- 
ments wliich are consistently rated by the judges arc combined in a 
qucstionnaiie. This is given to two groups of people, one “known" 
to be "favorable” and tbo other “knoam" to bo “unfavorable" on tlic 
topic involved. If tlie test scores discriminate between members of 
tho two groups, Uica the test is said to be “validated'' and Uie score 
to be a measure of the attitude. 

Since the test designers and j'udgcs have no explicit criterion for 
determining whether a question or statement is reievant to tho par- 
ticular attitude in question, it is not surprising that the items selected 
are frequently quite ambiguous. Consider, for example, Thurstone’s 
and Chai'e's test for “measuring” allitudes touard tho Church (1930). 
Here, not only was “attitude” left ill-dcfisciJ but the “Church" was 
Hut defined at all. In their item 26 (“I regard the Clmrcli os a para- 
site on society") “Church” is apparently used to mean “organiicd 
religion.” In item 31 (“There b much wrong in my Church, but I 
fed it is so important that it is my duly to help improve it") “Church” 
means “cillier the particular congregation 1 attend (if any)” or “tlie 
dcDoiniDalion with which 1 associate myself (if any) Item 34 (“I 
fed Uiat Church attendance b a good index of tho nation's morality") 
would be agreed to by a refigjous citizen of a capitalist country, im- 
plying tiiat his nation is moral; whereas an ardent Communist in the 
would also agree, but to him high attendance oicaos im- 
morality. hlany otlier amhiguilies exist ia tho items. For Uiis reason 
different tests of Uie same attitude may not always yield consistent 
results, a difficulty wliich could be avoided by proper definition. 

“Measures" of profit in business may also be quite meonsbtent, 
and for Uic same re.ason: profit is usually very poorly defined. Dif- 
ferent accountants can derive widely divergent profit figures from the 
same set of industrial transactions. 

For the reasons which fiave been considered, tficn, it is essential in 
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research that operational dcfimtions b« provided for all properties to 
be measured. Only bj’ so doing can the relevance of observations be 
assured and a basis for ad}ustment be provided. In the ideal opera- 
tions specified in the definition, the relevant measurement standard 
should be specified. For structural properties measurement standards 
are usually a matter of convention and hence do not have to be ex- 
phcitly identified as long as they arc knorvn. For example, the metric 
standard of length is the meter, which, by international agreement, is 
Uie distance between two scratches on a platinum-iridium bar kept 
near Paris. The standard ol time is the mean iolar second, which ii 
1/S6,-100 of a mean solar day. The metric unit of mass is the inertia 
of a block of platinum kept near Paris and is called U)C kilogram, and 
so on. In the United Stales the Bureau of Standards has reeponsi- 
bility for maintaining appropriate standards by calibrating measuring 
instruments and specifying the accuracy the}' roust be capable of 
yielding. 

Formal Properties of Measurement 

must be careful not to impute automatically to the numbers 
obtained by any process of assigning numbers to objects, events, or 
properties, the properties which these numbers have os numbers. We 
can add the numbers of two houses or of two car registrations, but the 
question is whether or not the sum has any meaning, and if so what. 
The numbers of two houses may represent some property of these 
houses, but the sura of these two numbeis may not represent any 
property of the two houses taken jointlj'. This is quite apparent m 
the case of bouse numbers but may not be so apparent in other cases; 
for example, the sum of the teinpcraUires of two bodies obtained on 
a Fahrenheit scale docs not represent the temperature of the two 
bodies taken jointly. 

The formal properties of a scale should describe what operations 
can be meaningfully performed on the properties which are measured 
on the scale. The scale of wright is said to be additive because we 
can find an operation (or set of operations) for joining two bodies m 
such a way that the weight of the union is equal to the sum of the 
weights of the bodies taken separately. It will be recalled that, in 
the earlier discussion of density, Campbell argued that such an oper- 
ation could not be found with respect to density. The formal prop- 
erties of a scale, then, do not describe properties of that which is being 
measured, but properties of the operations which can be performed 
on Uiat which is being measured. Thus, to say that the scale of 
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weight is additive is not to say anything about weight itself, but it 
docs say something about what operations can be performed on ob- 
jects which have weiglit. It is this fact that makes the evolution of 
measurement possible. If temperature itself were not additive, for 
example, the Kelvin scale uxiuld never have been developed. The 
Faljrcnheit and Kelvin scales ot temperature yield measures with dif- 
ferent formal properties. The fonnai properties of these scales repre- 
sent different seta of operations which can be performed on tempera- 
ture, and not dilfereacos m temperature itself. 

Put another way, when a procedure for assigning numbers to a 
property is developed, the properties of the procedure dictate the 
kmd of scale involved, and it is not tJic scale that dictates what pro- 
cedures are possible. Translation of the characteristics of measure- 
ment operations into formal properties of a scale employed is a safe- 
guard against our treating tlie numbcis oblamed in a way which 1ms 
no operational counterpart. 

Accuracy of Measurement 

A count, elasslhcstion, rank, or interval or ratio tncasurcmcnl has 
little slgsiUcaoce (if any) without some knowledge of its accuracy. 
All types of measureuieot are subject to error, and an estimate of the 
magnitude of this error is necessary in order to know whether or not 
the measures obtained arc usable in any specific pure, or applied, 
research situation. 

The value of meosurement can perhaps best be appreciated if one 
realizes that the range of uses to which measurement of a property of 
an object or event can be put increases wUi its exactness. A per- 
fectly accurate measurement of a property would make it possible to 
answer any question or solve any problem involving only that prop- 
erty; and, furthermore, it allows one to do it licariously. For e.v- 
ampie, if wo want to dclcnuine wliellicr wc can move a large carton 
through a doorway, we can, of course, just try. It is generally much 
simpler, however, to delenuine the magnitude of the shortest of tlic 
cation’s three dimensions and the width of the opening and compare 
them. Similarly, by measuring the dimensions of a table lop we can 
select a tablecloth in a store wiUiout having the table present. 

If wo know that Uic carton is lew than 3 feet wide, this will en- 
able u3 to determine that it can pass through doorways which arc .1 
feet wide or more. But it will not allow us to determine whether it 
will poss through doomays wliich arc Itu than 3 feet wide. If. on the 
other hand, we knew its exact widtli, we not only could make such 
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a determination for every doorway, but also could use this information 
to answer any other question in which this property of the carton is 
exclusively involved. 

Accuracy of measurements is not an end in itself. If all we want 
to know is whether we can move a carton through a 3-feet-wide door- 
way, all we need know is whether the carton has a dimension of less 
than 3 feet, not its exact dimensions. In pure science, where the ulti- 
mate uses to which a measure might be put by others is not generally 
known, becoming more accurate than is required by the question at 
hand can be justified on the grounds that someone else may require 
this greater accuracy in solving his problem or answering his question. 
In Chapter 13 wc will consider in detail how to detormine the degree 
of accuracy of measurement which is required in a problem context. 

There are several possible sources of error in measurement which 
may contribute to it separalefy or in combination. These are (1) fh® 
obseiver, (2) the instruments used, (3) the enNuronment, and (4) the 
thing observed. IVe will consider each of these in turn. 

Error due to the obser%'cr 

The obsers’er may not follow the icquired operations either because 
be is neglectful or because it is not possible practically to do so. Id 
this way he may bias the observations. In particular he may not 
make the necessary readings as accurately as possible. He may read 
dials or meters or rules incorrectly. He may not accurately record a 
response of a subject he has questioned, or he may fail to see what 
the respondent did. Bis senses as well as his powers of concentration 
are not perfect and hence lead to error. 

Round-robin tests, in which different observers of the same object 
using the same instruments obtain significantly different readings, 
provide strong confirmation of the fact that even in such “simple 
operations as counting the observer can be a source of significant error. 

Obsemr error first came to the serious attention of science when 
Bessel noted what he called the personal equation. In the nineteenth 
centorj' it was observed that in physical measurements the distribu- 
tion of an observer’s errors tends to form a normal cuive. The bi:^ 
and spread (characterized in terms of probable error or standard dcvi* 
ation) differ from obser,er to observer and for some observers fro® 
time to time. This distribution of obscn'cr errors, if gnea any atten- 
tion at all, is usually assumed to be nonnal in character « itliout any 
supporting evidence. 

How can Uie nature of observer error be detcrunned"’ The ans^^cr 
depends on the nature of the objeet or ei.'cnt observed. If the property 
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has a constant value over a long enough time to permit a large num- 
ber of obser\’alions, tlien multiple observations can be made and their 
diitribulion analyzed. First, it U necessary to determine tlie form of 
ihia distribution. This involves fitting a cmv'c to tlic data, a subject 
to be discussed in Chapter 8. Next the bias and dispersion of tlic 
distribution must be detenninud. iTlicae will be discussed in Chap- 
ter 7.) Tlie bias (deviation of Uie mean of the distribution from the 
“true” value) can be detenuinci}, boivever, only if tlie true value is 
icQOTvzi. Since this is never known in research, obsen'ers are usually 
checked by using a standard object or event under specified conditions. 
The standard object or event has prcxdously been measured, using the 
most highly contrulicd and careful observations possible. The mean 
of these observations is Uien aismned to be the true value and becomes 
the basis for computing observer bios. Tliis assumption is important 
and makes clear the fact that observer bias is never itself measured 
witliout error. 

If the thing obscpi'cd is destroyed or stgnificanUy changed with re- 
spect to the relevant property by (be observation process, then the 
procedure just described cannot be used. It is still widely believed 
that in such situations the determination of observer error is not pos- 
sible. This belief is not wcU founded. This type of problem arbes in 
riuality-control work where inspection or testing of an item often in- 
volves its destruction, or m social surveys where a respondent may 
be changed by the first interview. The metjiod for determining ob- 
server error in such situations is briefly as follows. The items to be 
measured (or a random sample • of them) are divided by a random 
process into groups of (approxiroalely) equal size. Then groups are 
assigned to observers at random. Each observer classifies or meas- 
ures the elements in the group assigned to him. The differences in 
llic distributiDns of tlie obscrv-alions are due to both observer error 
and the differences in the groups assigned. The probabilities associ- 
ated with differences due to U»e fonning of the groups can usually be 
determined, and these can be made as small as desired by increasing 
the size of tlie groups. Tliere differences can be "subtracted” from the 
total error, leaving the obicr>-cr error for examination. The different 
groups c.‘in also be a«ign<-d to the same observer at different limes if 
we are interested in his errors rather Oiaa in tliat of a group of ob- 
seivcrs. For tiie details of such an analysis see Hanson and lilorks 
(1958). 

It may be helpful to describe how observational errors of two dif- 

•See Chapter 7 tor a discussion of random sampling. 
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fercDt observers were measured in one particular situation. Tlic ob- 
Ecr\'ation involved in this experiment was of the auditing type. Serv- 
icc-crcdit-eompcusation forms of a company were filled out to corre- 
spond with their normiil usage but some errors were deliberately in- 
stTted on approximately 50 per cent of the sheets. Two auditors from 
the company were selected as subircta. Ilach auditor was given the 
same 120 forms containing 01 sheets vritti errors and was allowed 25 
seconds per sheet to chissrfy it aa correct or liaving an error. The 
auditors were Llicn given 190 fonns containing 90 sheets witli errors 
and were allowed 30 Ecconds per sheet Similar examinations were 
made of other batches of forms at 33, 10, *15, 50, 00, and 70 seconds per 
sheet. The performance times were &vcn to the auditors in ranciom 
sequence. It look Ecverul runs for the observers to setUe on an ob- 
servational procedure. These early sets of observations were conse- 
quently eliminated from tho analysis. The remaining daUi arc shown 
in Figure <5.2. 

From these data it is clear Umt the two objcrvers diftcr signifi- 
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cant]/, and jiatiftlical tests coatimicd tJits fact. Such cl«{a not only 
permit a scJcction ol observers but also indicate hon'much time should 
be nlloK'cd for an observation in order to obtain a specified level of 
accuracy. This experiment should not be interpreted as having gen- 
eral significaneo for the auditing function; it la used here only to il- 
lustrate how determination of observer errors can be made.* 

Error clue to uulrumcnts 

The instruments used in making obsera'ations, iikc the obscr%'cr, 
may be biased and/or bo inconsistent (variablci and hence bo cbnrac- 
tented by a distribution of errors. The procedure for determining 
tlicsc characteristics is the same as that for an observer. 

When an obscn’ation involves both an inslniincnt and an obscr\’cr, 
they can bo considered in combination and a joint observational- 
instrument error can be detennined. It is more difficult to obtain Uie 
error functions of each fcpanilcly, since neither can bo held constant 
wit)} respect to the error, but it can be done by Die vso oj designed 
ex|)rrinjcnts such as are discussed in Chajiivr lO- In surb situations, 
bonever, uc goneraily want to find Die best combiaotion of observer 
and instrument; t)iis can be determtoed by ar experiment nhich es- 
tafalishcs the error function for each corobination. 

The improvement of insCrufflcnU is an important way of reducing 
obsi'r>*atioDa( error arising not only from the inslrumcnta themselves 
but also from Uic obscrsxr, the environment, and Uie respondent. 
9ifcchanisation ol observation, for example, has contributed to tlic 
reduction of error in every branch of science. In astronomy, for ex- 
ample, mechanical means of timing the movement of a star across 
Uio field of a telescope h-os eliminated or at least significantly reduced 
the error due to variations in reflex time of human observers. Me- 
chanical iraffic counters have eliminated or significantly reduced er- 
rors due to failure of memory or carelessness of human counters. 
Hidden cameras and microphones and one-way glass have helped rc- 
ilucc observer influence on human subjects in psychological and social 
cxperimcuialiOD- iVntcIics nre available vshich are not affected by 
changes in temperature, moisture, atmospheric pressure, motion, or 
magnetic fields. One could go on indefinitely DnuineraDeg tlie con- 
tribuDon of iaslrumentaUon and meebanization to the reduction of 
error. 

It is iaspartant ta reaUse, benerer, iJwit so matter iew JygJjJy de- 
v elopetl these instruments arc they stUl cosiribute to error and tJiat a 
determination of tlie imignitude of this “eoatribution” is essenDal. 

*For dfeUils on this r«Kim.-h see Chamben and Clark (10£7). 
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Error due to environment „ 

The endilione under .hW. the obscrvnlions ■“JyScS, 
SO as to aiTcct the observer, the instrumcntfl, or the Uiing ’ 

and, o! course, they may tail to ratrepond to 
In order to determine how these dillcrenccs and variations affee 
aeeoraey ol observations it is neee^ary io know the Ians whiehjon^ 
ncct these ehanges to the observations. For example, 
lor variations in obsen ed lengU. ot a metal bar 'i™ f 
peraturo because the linear eoeHicient 0 expansion ol ‘h' 
us to eompule tlie resullinB chanses in IcnEth and lienee to adjust 

observations. ... . -t ond ad- 

In many areas ot inquiry sueb knowledse is not “'’“'“'y . 1 
jnstment of observations to compensate tor 

and ehanges is not possible. It is essential here to have at tot an 
estimate ot the resulting error. This may 
Only by such cxpcriracnlation can that knowledge be obtained 
will make adjustment for error possible in the future. 


Error due to the obscricd -Aiirca 

That which is obsened may be ciUicr an active or a . 

of error. First, the process of observation may lUe f ^ . 

havior of the obsen-ed so that we cannot observe iU natural pe 
tor. Such an effect of the process of observation on . 

observation has come to be charactcmcd as the tndefcrmina^ P 
ciple. In quantum mechanics, in which this principle was 
mulated, it appears to be possible to reduce the error in the 
tion of the momentum of a particle at the cost of increasing c 


in the determination of its location, and conversely. ^ 

The indctenrunacy principle has been the center of a major roe o^ 
olo^cal controversy in science since its formulaUon by H^sen e 
The issue involved concerns the question of whether it is in eren 
nature or is a property of our state of knowledge, concepts, an 
scrvational methods. There is good reason to believe the 

One can maintain that the significance of this principle is ^ 
points up the need for physics to cease thinking of such e cine 
as electrons as point particles. In one observational context c 
in which momentum is determined — the electron has no exact 
This does not mean that we have only inexact ways of eterm 
its exact position, but it means that the electron has no exact posi 
This seems to offend our feelii^ that all "objects" can be oca e 
some specific place at some specified time . But the new p 
quires that we reinterpret the concept “object" in terms dea mg 
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the way it is observed. In effect, an object in tlie new mechanics is a 
“state of nature’’ which is described statisUcaily, it is not a “particle 
of matter.’’ 

One aspect of tiie new pbystes is that lualter and energy are taken 
as functions of each Oliver, hlatter is no longer conceived of as a 
substance charged with or discharged of energy. Jleasurcmcnla in 
tins area today involve statistical states and hence are probabilistic 
in nature. The accuracy of these measures can be reduced without 
any limit imposed by "pnnciples of nature.” As Sf. Phiftips (1949, 
p. 199) has observed; 

It is true that we cannot follow the path of an electron as wc can that of 
a billiard hall; but the reason b that an ctcctrou is not a bilhard ball. Wate 
mechanics is coneemed precisi^y n-itJi a nave function describing the state 
of the system, and a wave equation which tells us how this function changes 
in time. 

For detailed discussions of the indeterminacy principle in physics- 
discussions wUch argue lliab continuous reduction of error is not pre- 
cluded by the principle — see Margeuau (1959) and McKnight (1959). 

Wicn the object observed is a human being, lie may cause much 
more difficulty (and deliberately so) than docs tlie electron, particu- 
larly when he is called on for verbal testimony. This problem has 
been treated at length in Uio literature of social-survey methodology. 
(See, for example, AckoIT (1953, Chapter IX). J Here we can only 
touch lightly on a few hlghlighU of (he subject. 

Two types of Imtruments arc used lo obtaioing verbal behavior 
from human subjects: tests and questionnaires. In a questionaaire 
it is assumed tliat the respondent possesses the iafonnatioa that tlio 
observer wants (e.g., his age); in a test such an assumption is not 
made (e.g., his intelligence). 

If a test-design procedure such as is described by AckoIT (1953, 
Chapter IX) is employed, the consistency of the respondent can be 
determined by uso of duplicate test items. Nothing in a verbal lest, 
however, provides a measure of the accuracy of Uie responses except 
in the case where tlie property under study is the ability to obtain a 
certain score on the test itself (c.g., in a rcading-abiniy or vocabu- 
lary test). In moat cases verbal test responses are substitutes for 
other types of behavior, direct obscrFalioB of which is difficult or 
costly to obtain. The definition of tho property tested, however, 
should indicate how such dirert observatbns miglit bo made. The 
reliance of the pure and applied bebaworal sciences on testa U so 
great that it seems wise to make a serious efiort to check the accuracy 
of lest results by use of a standard situation. An experimental check 
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on even a small sample of the iiopulalioa is likely to be very reveal- 
ing- Without such checks tlicic is no ground for Uic oft-heard asser- 
tion, "Verbal tests are Ujc best practical tool available.” Further- 
more, unless such evaluations of verbal tests arc made, we have no 
basis for adjusting llio test responses; Unit is, correcting for inac- 
curacy resulting from the instrument and the respondent. 

In using a questionnaire it ia assumed tliat the subject can give us 
directly the value of the properly being investigated. 

If a questionnaire ia constructed on tlie basis of an assumed but non- 
existent knowledge, Uicn, oecdk-ss to say, the results are worthless. 
Tliercforo, we should be careful, whenever we use a questionnaire, to 
examine aa beat we can tlic legitimacy of such assumptions. Tins 
kind of examination may he based ou previous studies or on a pretest 
deliberately conducted to check ihia assumption. 

Once we have decided that we can assume the subject to have 
knowledge of tlic points at issue, tlie problem is primarily one of com- 
auaication. This means that we want to make sure the subject un- 
derstands the questions as we want him to, that ho tries to give a 
truthful answer, and tiiat we get hU answer recorded accurately. 

At present wo have no organised science of linguistic communica- 
tion to iieip us in these problems, although there are contemporary 
attempts to construct such a science. jSce Cherry (1957).! As yet 
there are no general criteria to assist in Iho design of questionnaires' 
lienee, if we want to evaluate tJ>e adequacy of a questionnaire before- 
hand, in most cases the best we can bring to bear — other than con- 
trolled pretesting — is common sense and past experience. Past 
ence with questionnaires, however, is by no means meager. Numerous 
articles in the journals provide sound advice for specific types of 
questions. One of the best of Uiese is authored by Mauldin and Marks 
(1950). A number of the following remarks and illustrations are bor- 
rowed from this article. 

Response errors to questionnaire items may, in general, be due to 
two factora: (1) poor communication and (2) poor recall. ‘‘Poor 
commimication” may involve either (a) the failure of the question 
or the one who asks it to make its meaning clear, or (6) the failuic 
of the respondent to make his raeanii^ clear, or, if he makes it clear, 
the failure of the recorder of the response to grasp the meaning The 
failure of the interviewer to make the question clear may be due to 
his failure to imderstaad it. This, in turn, may be due to either poor 
training or lack of ability. 

It is obvious and trite to say that questions should use language 
familiar to the subjects, that they should not be too long, and that 
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there sl.ot,kl not be loo many of U,en, The problem, 
in the meaning o( “lanriliar,” "U» long,” and "too m.^- Se 
attempts liave been made to obtain Q-slematic knonlrfge on tl 
points by employing linguistic scales le.g, erm fnmiliaktv 

yet, the designer must rely primarily on 

with the population to he pnesUoncd, common sente, ^ P"“j 
H ll.ero is any doubt as to whether a question or statement con 
tains too much, it is belter to break it into two or more quesUons. 

Unlit recently Uie Eure.au of the 

liigbesl grade ol in /towering ,t 

lion should offer no difficulty for > .,j • ^ gfijje of school” 

correctly. However, many pcoide hwr ^Lol they “^ent 

amt ,L clLl worf in the gnesuon, that is. 

to.” Some people don t ”” .j, ..p-hat is the highest grade ot 

the word "completed . . ; tm ” nenindenl » asked, "Eld you tadi 

£iSr »' * r”- 

ilcm.^ CMauldm and Marks (W50, p- 6oO)3 
respondents tend to 

They may think 1”'“"“ ‘bi antieiplled. Tlic aullior, for eaample, 

nnUcipalwn should lUeli be ,„ultr\ ?” The rcspoawj waa, 

once SX%rdrot the Uckyard. 

"No," but it ihc housewife thouglit the quea- 

Furthcr questioning J ^ar *“ t ^ 

Uoa meant. "Do changed to "Do you 

for her own use. ,j^eer' Mauldin and Marks point out 

have any live poultry ‘ important to ask *Uow will 

that "in evaluating a qu ~ .,^^1 ^l,is question 

the respondent interpret this? than to osk 

mean?’” (1950, p. C50). -jjn^cd to the interviewer or ob- 

In some cases iho que i«dgmcnt concerning a property 

serve,; *.1 i«,J.« ^ idilion ol a honor, Ih. statu, ol 

ol that winch 13 j, ..,^„jiUon ol a liouso” or "slalus” 

a pereon). h u, notion as to wl.al Ibis 

is deflned, ^ it is desirable to bnrak tie ques- 

term means. " ?.„„„„enta which can be answered wathout 
lion down into a act 0 prejudices but from winch an 

involving the .uU»e«tly be derived. For 

accurate and « ‘ /Je.liOTS to determine the estent to wlueh 

TtS— rS. «... t. »b, "1. «.e supply =1 water 

adequalo?" 
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Errors oS recall may be due to lack of knowledge or poor memoiy. 
In one study, for example, P. P. Snutb (1935) asked studeala to check 
those books on a list (which he prodded) which they had read. The 
list contained a number of titles of nonexisiing books. But over one 
fourth of the students checked one or more of these. For other simi- 
lar results see Lucas (1940 and 1942). 

We cannot expect a respondait to remember very far back. When 
he says he has read a nonexisUDg book, he probably b not lying de- 
liberately; the title sounds familiar, and he concludes that he must 
hare read it, or he has doubts and decides to “play safe.” 

Response errors can be reduced in many cases by several alterna- 
tive design procedures. To do so may be costly, and the gain in ac- 
curacy may not be worth that cost. Consequently, 

In contidering the reduction of response error, the fundamental question is 
how accurate the data should be. Frequently an answer to this question wiU 
raise the further proUem of bow much it is worth (in dollars and in effort) 
to achieve a given level of accuracy. The level of accuracy required is, of 
course, dependent upon the uses to be made of the data. For example, 
much greater accuracy on age is required in preparing Ue tables for actuarial 
use than would be requited in clas^ying the population into age groups for 
an analysis of public opinion trends, [^fauldio and Marks (1950, p. 053}] 

We shall return to the question of what level of accuracy is re- 
quired is Chapters 3 and 23. 

Control 

Corjtrol of obserrational error consists of periodic checking to de- 
termine whether or not the magnitude of this error is changing over 
the period during which observation is made, during different obsert'a- 
tional periods, among different groups of observers, and so on. The 
nature of a control system and the critical methodological questions 
involved in the design of such a system will be discussed in Chapter 
13. 

It should be noted that the continuous reduction of error from 
other sources as well as observation is a major objective of science and 
is one of the principal measures of its progress. 

In Chapter 7, where sampling errors are discussed, we will consider 
a way of separating obscrv-ational error from sampling error and of 
estimating its magnitude. 
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MULTIDIMENSIONAL FEOPERTIES 

Somo properties cannot be represented by a single measure; two or 
more arc required. Such a requirement can be revealed by careful 
analysis of the meaning of the concept; that is, tJje way it is used. 
The “reliability” of objects or systems is such a property. For ex- 
ample, measurement of the reliability of light bulbs or components 
of an automobile may involve determination ot (a) average life of 
such components, and (t») the dispersion around this average. The 
"readiness” of a system is also a multidimensional property. For ex- 
ample, tlie readiness of a miKtaiy 8er%'ice may be defined in terms of 
(fl) Uie average number of pieces of equipment which are available 
for use at any moment of lime, (b) the dispersion of the number 
available around this average, (e) the expected length of use that 
can be obtained from equipment usable at any moment of time, and 
(d) Uio average time required to make operable any equipment that 
U not ojiorable at a moment of lime. 

Ideally wo nould like to amalgamate these various dimensions into 
a single measure of the properly involved. In principle this is pos- 
sible if we treat each measure as an objective and try to extract a 
single measure of performance from all. That is, altfiough ft may 
bo accessary to employ several measures to “capture” a property, it 
may be possible to develop a single measure of value of that property. 
The procedures discussed in Chapter 3 for obtaining measures of a 
decision's value are applicable in tfiis context. 


CONCLUSION 

It is clear from even a cursory examination of current research 
practices that measurement and me-asurca (end to be taken for granted. 
In problem and question areas where sdence has frequently trod no 
great risk is involved in such casualncss because common practice is 
likely to be based on sound evaluation (of the measures employed) by 
researchers in the past who established the conventions used in the 
present. As science moves into new areas, however, the carry-over of 
this unawareness of problems of measurement can lead to e.\tremely 
ineffective research. In psychology, for ejcarapfe, only in recent yearn 
{largely as the result of Stevens’ work) have researchers been giving 
measurement the attention it desen'es. As a result there has been a 
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Errors of recall may bo due to lack of knowledge or poor memory. 
In one study, for example, F. F. Smitli (1035) asked students to check 
those books on a list (which he provided) which tlicy bad read. The 
list contained a number of titles of noDcxtslitig books. But over one 
fourth of the students checked one or more of these. For other simi- 
lar results SCO Lucas (1940 and 1942). 

We cannot expect a respondent to remember very far back. When 
be says he has read a noncxisUng book, be probably is not lying de- 
liberately; the title sounds familiar, and ho coocludes that he must 
have read it, or he has doubts and decides to "play safe.” 

Response errors can be reduced in many eases by several altcma- 
Uve design procedures. To do so may be costly, and the gain in ac- 
curacy may not be worth Utat cost. Consequently, 

In considering the reduction of response error, the fundomcnUl quKtion is 
how accurate the data should be. Frequently an answer to this question triU 
raise the further problem of bow much it is worth (in dollars and in effort) 
to achieve a given level of accuracy. The level of accuracy required i», of 
course, depcndcat upon the uses to be made of the data. For example, 
much greater accuracy on age is required in pTcparicig life tables for aetuanai 
use than would be required in elasdfyiag the population into age groups for 
an analysis of public opinioa trends, (hlauidm and Marks (1050, p. 6^3)1 

We shall return to the question of what level of accuracy is re- 
quired in Chapters 8 and 13. 

Control 

Control of observational error consists of periodic checking to de- 
termine whether or not the magnitude of this error is changing over 
the period during which observation is made, during different obsen a- 
tional periods, among different groups of observers, and so on. The 
nature of a control system and the critical methodological questions 
involved in the design of such a system will be discussed in Chapter 
13. 

It should be noted that the continuous reduction of error from 
other sources as well as observation is a major objective of science and 
is one of the principal measures of Us progress. 

In Chapter 7, where sampling errors are discussed, we will consider 
a way of separating observ'atioQal error from sampling error and of 
estimating its magnitude. 
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MULTIDIMENSIONAL PROPERTIES 

Some properties cannot be represented by a siaglo measure; lire or 
more nre required. Such a requirement can be revealed by careful 
analysis of the meaning of the concept; that is, the way it is used. 
The "reliability” of objects or systems is such a property. For ex- 
ample, measurement of the reiiabdity of light bulbs or coro 2 >oncnts 
of an automobile may involve determination of (a) average life of 
such components, and (b) the dispetaon around this average. The 
“readiness” of a system is also a multidimcnsiooa! property. For ex- 
ample, the readiness of a military scni-ice tn.ay bo defined in terms of 
(o) the average number of pieces of equipment which are available 
for uso at any moment of Umc, (b) the dispersion of the number 
available around tills average, (c) the expected length ot use that 
can be obtained from equipment usable at any moment of time, and 
(d) the average time required to make operable any equipment that 
is not operable at a moment of time. 

Ideally ire n-ould like to amalgamate these various dinjensions into 
a single measure of the properly involved. In principle this ia pos- 
sible if we treat each measure as an objective and try to extract a 
single measure of performance from all. 'Hiat is, alUiougb it may 
be accessary to employ several measures to “capture" a property, it 
may bo possible to develop a single measure of value of tiiat properly. 
Tlie procedures discussed in Chapter 2 for obtaining measures of a 
decision's value are applicable in (his context. 


CONCLUSION 

It is dear from even a cursory cxaiiunalion of current research 
practices that measurement and inc.asure8 tend to be taken for granted. 
In problenr and question areas where wicnco has frequently trod no 
great risk is involved in such casuolness because common practice is 
likely to be based on sound evaluation (of the measures employed) by 
researchers in the past who established Uie conventions used in the 
present. As science moves into new areas, however, the carry-over of 
this unawareness of problems of measurement con lead to extremely 
ineffective research. In psychology, for example, only in recent years 
(largely as the result of Stevens' woifc) have researchers been giving 
measurement the attentioD it deserves. As a result there has been a 
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noticeable tiglitening of procedures, bringing with it a scientific ma* 
turity that reduces the defensiveness which the psychologist must 
display before the physical scientist. Measurement in applied phys- 
ical science is likely to be as loose and ill conceived as in any other 
branch of science. For luany of tlie very important properties used 
in cn^nccring tlicrc are not yet available either adequate definitions 
or effective measures. This is particularly true in inspection and 
quality-control procedures, where the relevance of the tests employed 
and the measures derived from them are frequently ill cstafalished- 
Xfcasurement, pcriiaps more than any other rcscarcli actirity, has 
been the principal stimulus of progress in both pure and applied 
science. 
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SAMPLING 


review and freview 

It m>y te holplul at tUU point W mvicw bticlly 
lias been covered and to preview wl»al is to come m Uie ncx 

AUer an introauctoo' aUcusslon ol llic nature “”j ii,e 

meUioJa (Chapter 1) we coDsWetea Uic nature ol “ j-,,. 

criteria for a hcsl solution to it (Chapter In C p ,j ^ gg. 
cussed procedures for idenlifjins the coalrellcd and "n'™^ ” ^ g 

pert, of a prohlem (X, and l’,). and proerdures 4 

measure of the value of the outcome of a decision ( 
was concerned with the symbolic representation of the prooic 
ation in a decision model in which the measure of performance i i 
related to the variables X, and Y, by a function /; that is, w 
sidered construction of a decision model of the fonn: 

r = /(.v..y,). 

In Chapter 5 we discussed definition of the components of a m^^ 
so as to assure relevance of the observations that arc ma ® ^ ^ 

to determine their values. Then, in Chapter 6, we j 

more detad how the observarions sliould be made so J 

sj-mbols (which we called measure#) that could be used m 
decisions involving the concept represented. 
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Now, ia order to evaluate an uncontrolled variable or constant 
(i.e., parameter), we usually must determine what subset of the set 
of possible relevant obseiwations should actxially be made. That is, 
wo roust design the Bample of elements wliose properties are to be 
obscr\'ed. The design of samples is the subject of this chapter. Next 
wo consider how to combine the data obtained from a sample into an 
estimate of the relevant property (Chapter 8). In gome cases an 
estimate may already bo })T 0 vjded by past observations or by a tJjeory. 
Then the problem is to determine whcUicr or not the value Uius pro- 
vided should be used. This problem, that of testing a hypothesis, is 
discussed in Chapter 9. Tesla of hypoUicscs may also be used in 
selecting among two or three allcrnaUw courses of action or in deter- 
mining whether or not a iwramclcr is relevant; that is, wljetiier it 
significantly affccta tlic outcome of a decision. Testa of relevance by 
eTpenmentaCian, and regression and correlation analysis, arc Uie sub- 
jects of Cliaptcr 10. 

Tliis and Uio next three chapters deal with the various a8i>cct8 of 
statistical inference, largely from the point of view of what has 
come to bo called statistical decision theory. No attempt is made 
here, however, to provide comprehensive coverage of the field of 
statUtics. Tiie objective is to analyze the research decisions that 
ore involved, to construct models of them and derive solutions where 
possible and, where not, to go as for as we con in this direction, 
The lack of technical details in this treatment is likely to bother 
the professional statistician; the presence of what technical detail 
there is, is just as likely to bother the novice. I fear that the median 
I have struck znay not be a happy one, but I hope that it is useful. 


INTJlODUCnON 

ItTienever the definition of a concept ret^uires that observations 
be made of either (o) a number of dements in a population, or (b) 
an dement in a number of different situations, two questions arise: 

(1) How many observations should be taken? Tliis is a question 
of sample sire. 

(2) Which subset of elements or environments should bo selected 
for observation? This is a question of sampling design. 

Sampling, then, is concerned wiUi determining uliat and how many 
observations arc to be made where it is not possible or prsctical to 
make all the observations that are ideally desirable. 
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The techniques of sampUiig have dcvclojMjd very rajiidly in the last 
quarter cenlurj*. Knowledge of tlicsc techniques is still not widely 
disseminated among scientists. For this reason it will bo necessary 
to review at least some of Uns Ixxly of knowledge preparatory to dis- 
cussing the mcliiodological pruhlcins which arise in connection with lU 
This review is not inlcndeil as a substitute for a detailed study of 
the field; its purpose is merely to provide enough background to allow 
us to discuss meaningfully the inelhodological problems tiiat arise 
within sampling. Detailed information on sampling tccliniqucs can 
be found in Cochran (1953), Doming (19(3), Hansen, Ilurwitr, and 
Itfadow' (1953), and Yates (1919). 

It is difficuU, if at ail {wssiblc, to discuss sampling infh'jwndcnUy 
of estimating procedures. For Utjs reason wo shall present what is 
required of estimation theory without comment or question in this 
chapter. In Uic next chapter we will discuss estimation thcor>' in 
detail and bring the methodological problems of Bampling and esti- 
mation together. In tliia chapter, however, we shall try to bring 
together Uio problems involving observational and sampling error. 

To facilitate this exi>osiUon it will bo helpful to deal with a "toy 
population." Suppose that Uie population consists of six ciiildrcn 
whose ages arc 2, 3, 4, C, 9, and 12 years. The total of tlicso ages 
is 30 years; the average is 6 years, and Uic range is 10 years. Our 
task is to use obfcn-ations of a sample of these ehildrea to estimate 
the average age of the group. 


ACCURACY, DIAS, AND PRECISION 

Each possible sample (which b drawn williout replacing the ele- 
ments) and its average arc shown in Table 7.1. 

For tlic moment consider only samples of two. Only one of these 
samples (3,9) has an awage equal to that of the population. Each 
of the other samples has an average which differs from that of the 
jiopulation, and these s.an)ple averages are inaccvrale estimates of the 
population average. The inaccuracies consist of overesfinmfes and 
underestimates of the population average. 

Kow assume that we have a way of drawing samples of two from 
this population so that each possible sample has the same probability 
of being drawn. Tlicn, if we use tliis sampling procedure and the 
sample a^'crage as an estimate of the population average, tfie expected 
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value • of such a sampling-csUmating procedure would be: 


p + 3\ 

1+ ‘ (-+') 1 


A 2 J 


^I5\ 2 J 


la this case the expected value (EV> is crjual to the true value (TV). 
The difference lEV -• TV) is the bio* of sucli a procedure, which in 
this case is zero. Ileacc this procedure i* said to be unbiased, "^oic, 
howe\er, tliat an unbiased procedure con yield inaccurate estimates. 
In this case it is more likely to do so than not. 

The bias — or lack of it— of a sampling-estimation procedure is 
independent of Uie sample size, as can be verified by examining the 
average of the a\cragcs from Table 7.1, assuming that each gaini>!e 
is equally likely. 

N’oip consider the difference between the lowest and the highest esti- 
mate yielded by each sample size. These range* of estimates arc 
shown b Table 72. 

Table 7.2. PaEasiON or Vabiovs Size Sauples niou "Tot” Pofi'LATtoN 


Saiople size 1 2 3 4 & 

Iton^ of Routes of meut 10 S 6 1 2 


It is apparent that the esUmatca become le&s voridble (or more pre- 
cise Of reliable) as the eautple size inercasc-s. 

The most commonly used measure of di.-pcreioa of estimates is the 
standard error, which is the square root of the mean of the squared de- 
viations of the estimates from the tree value. The standaid error of 
estimates of the mean, is 



where £ = the estimates of the meaD based on a sample of specified size. 
M — the true value of the mean. 

fc = the number of posrible distiiict samples of the specified size- 

Hence the standard error of a set of estimates is its standard dceiation. 
Its square, is the ntnance. 

In order to simplify this discussaon we use “per cent of estimates 
withm 1 of the true value” as a measure of dispeision- 
’The arenge of the STcragcs u tbe muober of samples approicbes mfiaity. 
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SIMPLE n^VNDOM SAMPLING 

Tho determination of bias and standard error presupposes that each 
possible sample has a known probabitity of being drawn. It we drew 
the elements in such a way that each clement had an equal opportunity 
of being drawn, then this condition n'ouid be satisfied. In practice 
these equal-probability conditions are approximaied by uniquely 
numbering each element of the population and using a table of random 
numbers * to select llie numbers and, hence, the elements. This most 
common tj-pe of sampling is called cither simple or wtrczlndcd 
random sampling. 


SYSTEMATIC EANDOM SAMPLING 

^’o\v suppose that the clemcnU of the population or symbols rep- 
rcseatHig them are in some phj'sical order; /or example, people stand- 
ing in line, names listed in a book, items on an assembly line. Each 
element must be uniquely identifiable by iU location in the sequence 
Now suppose that there ate iV clemcDta in Uic population and that wo 
want a sample of sire n. The "sampling ratio” is iliea N/b. 1/ this 
ratio is not an Integer, it is rounded off to Uio closest one; call this 
integer m. Now wo can select a number from 1 to w at random; let x 
represent this number. Then we can select a systematic (random) 
sample by selecting those elements in the iollowing positions in the 
sequence: 

r,x + m,x + 2m, •••. 

If AT/n is so integer, then each possible sample will be of equal size. 
Othenvise some samples will contain one more element than others. 

Suppose tliat there arc JOOO members of a population for each of 
which there is an associated index card numbered from 1 to 1000. If 
we want a sample of 100, tve select a number at random from 1 to 
^®'’9ioo ~ Suppose that it is 7. Then the sample would consist 
of the members of tJie populntion numbered 7, 17, 27, • • *, 997. 

For any systematic random sample an unbiased procedure exists for 
estimating tlie mean and other characUrisUcs of the population. The 
variability of these estimates may be greater or less than those yielded 
by simple random sampling, depending on the way the population is 


See RAND (1955). 
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ordered. It Oic onltrinR is randuin, tlicii systematic nuidom sampling 
yields the same varialdlity of estimates as ilocs suiiplo random sain* 
pling. 

Suppose that the toy ])opuliktion is ordcrctl in the following way: 
2, 3, 1, 0, 0, and 12. Tljcre arc two possible systematic samples of 
3 elements; 2, 4, 9 {average « 5) and 3, 0, 12 (average » 7). The 
range liere is equal to 2. If Uic i>opulation order is 2, 0, 3, 9, 4, 12, 
then the two possible samples of 3 clcmcDts arc 2, 3, I (average = 3} 
and C, 9, 12 (average ~ 9), and the range is 0 (Uie same as that of 
simple random sampling) . Since the standartl error of estimates bsied 
on a systematic sample depends on population order and the essential 
characteristics of this order arc often not knoni) beforehand, nceu* 
rate estimates of standard mors are diOicull to obtain without some 
preliminary data about the population. 

For detailed discussion of systematic sampling sec Madow (19U 
and me). 


.MULTISTAGE SAMPLING 

Where the population to be sampled !$ (or can be) divided into sub* 
groups which arc exclusive and cxJtaustive, it is possible to sample in 
stages. Tire subgroups would be involved in the first stage of sclec* 
tion, and the elements in the second stage. At either stage wo can 
take citlicr a complete count or a random (simple or systematic) 
sample. If wo take a complcto count at botl) stages, we have, of 
course, a complete count (100 per cent sample) of the population. 
Then Uicrc are three types of sampling designs for two-stage s.amphng. 
These are shown in Table 7J. 

There may be as many stages of sampling as there arc levels of 
classification. In sampling involving tliree or more stages it is pos- 


Tabls 7.3. Tttes or Two-Staob Sauples 
First Stage Second Slago 

(Selecting Groups) (SclecUng Items) Name of Sampling Ftan 


Sample* Sample* Two-stage randoiu 

Complete count Sample • Stratified sampling 

Sample • Comidetc count Cluster sampling 


* Simple random sampUng or a vaiiatioa thereof. 
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Table 7.4. Possible Two-Staob Rasmu Samples op Two Elements nioif 
Two OP TiincE PiuMABT Sampuko Units: 2, 12; 3, 9; and 4, 0 


Siunple 

Sample Average 

Sample 

Sample Average 

2,3 

2.5 

12, 1 

S.O 

2,y 

5.5 • 

12,0 

9.0 

2,4 

3.0 

3,4 

3.5 

2.0 

4.0 

3,0 

4.5 

12,3 

7.5 

9,4 

6.5* 

12,9 

10.5 

9,6 

7.5* 


37% f 


* Deviates bjr no more tiun 1 unit from the population aveiage. 
t Perccnlage Of sample averages that deviate by no more tlian 1 unit from tlie 
population average. 


slblo to siT&tily at any stage before the last aod to duster at the last 
stage. Such a combuiatioQ yields a stratihed cluster sample. Uolts 
sampled in the first stage are called pritnary; tijose sampled in tJie 
second stage, eecoaderp; and so on. 

Multistage liandom Sampling 

In Ibis type of sampling a random (simple or systematic) sample 
is taken at each stage. Unbiased estimated of population character- 
istics can be obtained from such sampics. The standard errors of such 
estimates can never be less than that obtained from simple random 
sampling. Tlicy can be larger, depending on the kamoyeiieity of the 
groups; that is, the similarity of the groups with respect to the prop- 
erty being investigated. Hie more homogeneous tlie groups the less 
variable are the estimalca. See Tables 7.4 and 7.5 for such a com- 
parison, 

^Vhero Uie value of the property under study is correlated wiUi tlie 
size of the population (e.g., the total weight or income of a popu- 
lation), variability of estimates can normally bo reduced by making 
the probability of selecting any subgroup of the population pro- 
portionate to its size; that is, by making tliis probability equal to the 
ratio of Uia group size to the population size. 

The principal advantage of multistage random sampling over sim- 
ple and systematic random sampling is Uiat every element of the 
population do« not have to be numberwl; only those elements in the 
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Taule 7.5. PossiBu: Two-Stage IUsdOU Samples of Two EuEUE-vra fbow 
T wo OF Tueeb Piuuabt Sautuko Umts; 2, 3; 4, 0; a-vu 9, 12 


S.miplo 

S.ajnp!o Avcfago 

Sample 

Sample Average 

2.4 

30 

3,9 

C.O* 

2,6 

4.0 

3,12 

7.5 

2,2 

5.5* 

1,9 

G.5« 

2, 12 

7.0* 

4,12 

8.0 

3.4 

3.5 

0,9 

7.5 

3,G 

4.5 

6.12 

9.0 

33% f 


* Dc\iates by no moro Uian 1 unit from UkC {lopulAtiun avcrugc. 
t Percentage of eamplc avcroseaUiat deviate by no luoro Uian 1 unit from the 
population average. 


groups which have been icicelcd in Uic previous stage need to be oum- 
bered. Furthermore, if the groups are located in different places, 
sucli a sample will generally require less travel than Ujc simpler type* 
of sample. These savings in cost may permit a larger sample to bo 
taken so that either more precision is obtained at tbo same coat or Uic 
same precision is obtained at less cost. WheUicr or not citlicr of these 
possibilities is realircd depends on tlic relative costs of observation 
and of preparing the elements for observation (e.g., numbering) ■ 

StratiEed Sampling 

Stratified samples may yield unbiased estimates of less or more 
variability than the preceding types for the same sample sizes, de- 
pending on the homogeneity of the strata and the division of the 
sampled elements among the strata. In general, samples of equal 
eize drawn from each stratum yield more variable estimates Uian do 
samples which are proportionate to the size of the stratum. This is 
seen by comparing Tables 7.6 and 7.7, Strata samples of equal sire 
allow us to make more precise comparisons between the strata, but 
this advantage is obtained at the cost of more variable estimates of 
population characteristics. 

The more homogeneous the strata the less variable are the esti- 
mates. This is seen by comparing Tables 7.7 and 7 8. If we have 
information on the homogeneity of the strata (in the form of the range 
or variance of the property characteristics among the member cle- 
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Tadle 7.6. PC(SS)HLE SrRtnnED Samfees of Tiuieb ^LE.^^E.v^a 
rnu\i TUB Stbata. (2; 3) and (i, 6, 9, 12} 


Sample Estuiwtftl Average t Sample Estimated Average t 


2.3,4 

35 

2, 0, 12 

fi.7* 

2,3,0 

4.S 

2. 0, 12 

7.7 

2,3,9 

ft.3* 

3,4,6 

43 

2, 3, 12 

a.s 

3,4.9 

6.3* 

2,4,6 

4.0 

3, 4, 12 

6.3* 

2,4,9 

50* 

3, 6, 9 

C.O* 

2, 4, 12 

6.0’ 

3, 6, 12 

70* 

2, «, 9 

5.7* 

3.9. 12 

S.O 

50% 

Deviates by no uk 

ire than 1 unit from Ute population i 

IV crogc. 


t To obtain an unbiased estmiateof Uic imputation average in dispro])ortiooalc 
etratiSed soinpiingi the sample average c.Tanot be used. An unbiased estunute 
con bo obtained by taking die average ol tlic sample from esdi stratum, multi* 
pll'ins it by tbc suo of Uie stratum, and dividing tlie sum of these values by the 
sUoof theiMpuUlion. For example, in the first sample listed (2,3,4) tlieesliroato 
is compub^ os follows. 


Table 7.7. Possible PnopontioXATB Stratiftes Samples op Thuee 
Eixmekts raoM tub Strata: (2, 3) and (4, 6, 0, 12) 


Sample 

Estimated Average 

Samplo 

Estimated Average 

2. 4,0 

4.0 

3,4,0 

4.3 

2,4,9 

SX)* 

3,4,9 

5.3* 

2. 4, 12 

0.0 » 

3, 4. 12 

6.3* 

2,6,» 

5.7* 

3.0,0 

0,0* 

2, 6, 12 

6.7* 

3,0,12 

7.0* 

2, 9, 12 

7.7 

3.9, 12 

8.0 

07% 


Deviates by no moie tliaa 1 unit from the population average. 
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Table 7.8. Puoportionate Stratified Samples or Three Elements 
FROM THE HeTEROGESEODB STBATa: (2, 12) AND (3, 4, 6, 9) 


Sample 

Estimated Average 

Sample 

Estimated Average 

2,3,4 

3.0 

12, 3 , 4 

6.3* 

2, 3,6 

3.7 

12, 3, 6 

7.0* 

2, 3.9 

4.7 

12, 3, 9 

8.0 

2,4,6 

4.0 

12, 4, 0 

7.3 

2, 4,9 

5.0* 

12, 4, 9 

8.3 

2,6,9 

5,7* 

12, 6, 9 

0.0 

33% 


* Deviates by no more than 1 unit from the popuktton average. 


raents), this infonoation can be used to obtain furtlier reductions in 
variability of estimates by means of what is called optimal allaeatton 
of saroplo size to strata. For example, if we have two strata contain- 
ing ni and nj items, with ranges flj and Jlj, respectively, the propor- 
tion of the sample allocated to the 0rst stratum is 

niRi 

»*iRi + njRj 

Correspondingly, the proportion of the sample allocated to the second 
stratum is 

712^2 

tiiRi + 

For example, suppose that the toy population is divided into t \'0 
strata, (2, 3, 4} and (6, 9, 12); and we want to allocate optimally n 
sample of four to these strata; for this case both and vi arc etjua! 
to 3 and Ifx = (4 - 2) = 2, and R, = (12 - 6) = 6. Then the pro- 
portion of the sample to be allocated to Uie hrst stratum is 

(3) (2) 1 

(3)(2) + (3)(G) “ 4* 

The proportion of the sample allocatwl to tlic second stratum i® 

(3)(C) 3 

(3){2) + (3K6) “ 4 
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Table 7.0. OniMfsi Alu}catiu:< (Aitiu)xihate) or a Sample or Four 
Elements pkou the Sibata; (2, 3, 4) and (C, 9, 12) 

Estimated 
Sample Average 


2, 6, 0, 12 5.5 • 

3.6.9.12 64) • 

4. 6. 9. 12 0-5 • 


• Deviates by no more Uian 0 5 unit from the populatbn average. 


Hence, the number of elements to be selected from the first stratum 
’3 1^(4), or 1, and the number of items to be selected from the eec^ 
ond Btratuiu is % (4), or 3. Table 7.9 shows the results obtained from 
using this method. 


Cluster Sampling 

la general this type o/ s.-)m|>lJng (a complete count of subgroups 
sampled) yields uobiased estimates of greater variability tbaa aoy 
of the other types discussed. Its advaotages are as follows. (1) II 
clusters arc geogrcpltieally dcliocd, cluster sampling generally yields 
lorvest sampling cost per observation. (3) As to multistage random 
sampllog a complete listing is required for only those groups selected 
for the sample. (3) Characteristics of (he clusters which may he 
relevant can be estimated with no variability due to sampling. {4) 
A cluster sample may be used over and over again because the sub* 
groups rather than elements are selected, sad chants of elements ia 
these subgroups allow us to estimate changes in the population os a 
wliole. 

The principal practical difficulty in the use of cluster sampling 
arises out of the fact (hat eacJi individual In the population must be 
capable of being uniquely assigned to one and only one cluster. This 
requires effective identification of individuals and clusters and the 
relationship between them. The assignment of individuals to clusters 
need not be done beforehand; in fact, it is usually done at the time 
of observation. But this may be a difficult task, For example, in the 
UB. Bureau of the Census "Post-Enumeration Survey” conducted in 
1950, each individual in t|je population of the United States had to be 
potentiafly assignable to some lirbg quarters as of April 1, 1930. 
This w.as extremely difficult to do, for it invxjlved the complex concept 
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"usual place o! residcuce." For eaample, it U Irecueutly hardj_ 
assign a person who travels in his busings to a usual ^ 

dence Yet this transient portion of the population is p 
td each member of it moat be aaeieoable to eome oue elustc • To 
avoid errora m this aseisumeot, the concept of usual Pl»“ 
dence” and related notions had to be carefully defined m 
with the principles given in Chapter 5. nhtained by 

Determination of population characteristics from ^ 

a cluster sample is not a sample statistical procedure. ^^ts 

„1 adjustment, of the data am rcqnW. To mate 
expert staUstical assistance is usually required A. discussio 
these technicalities can be found in Deming (1045) and Ma 
(1941). 


REPETITIVE (OR MULTIPLE) SAMPLLNG 

Dp to this point in the discussion of sampling hy Sg 

Sion as to how to draw the sample at any stage is ^ * 

any stage of the sampling is perfoirocd. It is also P08«‘^^®' ^ 
to draw one sample from a population, analyse it, and 
ing information in designing a second sample from the sam p 
lation. Such a procedure is called double somplinj. p' 
pling the data obtained from the second sample may be us 
neclion with the data from the first sample to improve the e^ien ) 
of estimaUs of population characlerisUce. For example, , 3 , 

from the first sample can be used efficiently to stratify tne p P 
lion or to allocate the sample to strata. _ nircavs 

In nonrepetilive or single samples a definite deciaon is 
reached only when all the stages are completed. But m 
sampling, according to Deming (1950, pp. 543-549) • 

... A small sample would first be taken: if the results are decisive, 
ther investigation is reqmred. But if the results are not Qecisi«. 
sample would be taken. The results of the first sample would neany ^ 
provide the necessary estimates ... by which the „« pro- 

very economically planned and made neither too large nor too smau 
vide definite and sufficient evidence for a rational decision. 

After a small first sample is taken, if the results are not J 

the next sample taken may also be small. The results 0 . ^ 
and second samples can be combined. If these are not ecuiv , 
other sample can be taken, and so on. Each subsamp^ °^'*^tatioiis 
selected, for example, that it constitutes a day’s work. Comp 
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can then Ic made to deterinme whether another sample is required. 
Such a procedure is called group tequeniuU sitrnpling. It can be used 
profitably where the treatment of Uio data is relatively simple and 
additional samples can be drawn very quickly or arc prepared before- 
hand. 

The method of sequenUaJ satojdiog just discussed suggests the 
following question.' would it bo possible to make only one observation 
at a timo and decide whctljcr or not the rvsulU up to that point are 
decisive? Such a scquciilial-smupliug procedure is feasible and is In 
use. In this type of sequential sampling the size of Uic sample is not 
determined In advance, but tlic rcsulU of each observation are used 
to determine whether an addiUonal observation is necessary in order 
to come to one of a set of specified eonciusions. On the average, 
fewer observations arc required by this procedure than by single or 
double sampling relative to a fixed error. In some cases only half the 
sample size required in simple random sampling will yield equally 
good results if sequential sampling is used. 

The sequential procedure cannot be used in all cases. Tbe following 
genera] conditions arc sui&cicot for iU application: (o) random sam- 
pling can be conducted for any sample size or for subgroups of sam- 
ples, and (b) tlio nuuibcr of obscrv'ations can bo increased indeflaitely 
at any stage of the procedure. Condition (o) fads to apply in tliose 
cases where we doubt that a very small sample can even approxi- 
mately represent the population. That is, in general, wherever strati- 
fication or multistage sampling is desirable, sequential sampling is 
not an cfiicient procedure to use. For example, if we were to examine 
a number of persona for their preferences with respect to some article, 
we would want to make sure that wo bad examined several profes- 
sions, several levels of income, and so on, before we came to any con- 
clusion; licnce, we would not apply sequential sampling, at least not 
until a minimum sample bad been obtained. Condition (6) fails if 
it is not possible to draw additional items into a sample after the re- 
search has begun. Tins is true of most social suncys, since it is usu- 
ally very difficult or expensive to increase the number of subjects 
during tlie course of the survey. 

Despite these restrictions, tlicw are many cases in whicli this method 
is applicable even in the social sciences. For example, in one project 
it was necessary to determine nbelber the perceniage of inmates in 
a state xocDJnJ institution who had a certain characterislio was sig- 
nificantly different from 1.5 per cent. For each patient there was a 
large envelope on file which contained, among other things, his medi- 
cal history. This history was generally quite long and required con- 
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siderable time for a complete reading. In such a case, where the 
sample was drawn from the file and could be increased with no great 
difficulty, sequential methods were extremely efficient. 

In general, sequential procedures arc applicable only where tlio 
observations can be made one at a time or in groups without consider- 
able loss of time between observations. This means that in most 
cases the subjects or data concerning them should be concentrated 
in a relatively small area and hence be easily accessible. The individ- 
ual subjects must be uniquely identifiable so that there is no ambiguity 
about which one is to be selected in each drawing. In effect, this 
method requires the ability to manipulate efficiently the sample el®' 
ments, whether these be records, individuals, or groups. It is apparent 
that satisfactory sequential-sampUng conditions are most likely to be 
encountered in research in which records or very small concentrated 
populations are involved. 

The analysis of data obtained by sequential sampling is different 
from that applied to noasequentiat-sampiing procedures. A discus- 
sion of analyses applicable to sequential procedures can be found m 
Statistical Research Group (1940) and in Wald (1947). 


JUDG^^ENT SAMSUNG 

The probability-sampling procedures which have been discussed up 
to this point involve either a complete count or a random sample at 
each stage of the procedure. Because all these procedures have been 
based on random sampling, it is possible to determine for each pro- 
cedure the probability that an estimate (which it yields) of a popu- 
lation characteristic deviates by any specified amount from the true 
value of the characteristic under investigation. This is not true for 
all methods of sampling at the present time. 

In some cases there arc well-defined subgroups of a population 
which seem to be representative of the population ns a whole, lu 
other cases practical considerations seem to preclude the use of prob- 
ability sampling, and the researcher looks for a representative sam- 
ple by other means; that is, he looks for a subgroup which is typi®u^ 
of the population as a whole. This subgroup is used na a "barometer 
of Uic population. Observations ate then restricted to this subgroup, 
and conclusions from the data obtained are generalized to the tola 
population. This is a type of conuson-senso procedure. For exampl®, 
for a number of years many people in the United States believed with 
respect to presidential elections that “as Maine goes, sa goes the 
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nulioii.” Tiiis c.\pri>s(il a runvicUoii at llic liiiiii Llmt Maine's voting 
bcliavior was tyi)ipul of Uial of Uic nuUoii as a whole An election 
forecaster who restricted his observations to Maine would have taken 
a jxijfjmcnl (or purposive t sample. 

Judgment sampling is very precarious, bee.iusc mucli stronger as- 
sumptions must be m.nde about the population luid tlic selection pro- 
cedure tlian arc rtiiuircd in probability sampling. For example, to 
select a typical city in a study of consumer purchases for the nation, 
the researcher may look for a city whose income distribution is simi- 
lar to tliat of Uie nation as a whole, whoso industries are typical of 
those of the nation as a whole, and so on. In cSect, he selccU a com- 
munity whicli is typical wuth respect to a set of properties, A, D, •••, 
•V, from which he assumes that Uic community is typical with respect 
to the characteristic X nhieli he is investigating. Now it does not 
follow that, because a city is typical will* respect to A, B, •• •, N, it 
is typical with respect to X, unless A,B, •••, N completely detennbe 
X, It is not enough to claim Uiat they arc highly correlate. Even if 
the correlation is high, the city selected may not bo typical with re- 
spect to the way X is related UtA, B, •••, X. Hence the further as- 
sumption is required that in the city sefected the rclatieosliip between 
A, D, iV and .V is typical. Assumptions such as these are likely 
to bo very doubtfuf, and their validation is likely to cost more tijen 
the use of a probability sample. 

Judgment sampling has another si^ificant shortcoming: sampUng 
errors and biases can be computed only if there is available a record 
of estimates derived from such samples in the past together with the 
true values that were being estimated in each case. In such cases the 
sampling error and bias can be determined coipirically, but there is no 
assurance that future use of such samfdiDg wili yield results consistent 
wiUi tJicso computations. Despite its ehortcomings judgment sam- 
pling does have its uses. Deming (lOGO) cites three. First, 

If a sample roast bo confined to only 1, 6, or 10 umts (Uorks, trade, cilics, 
counties, farms, pieces of material), ft judgment Eamplo would be preferable 
to a probabUity aample. In such very small samples, the errors of judgment 
are usually less than tlie random erwro of a probability sample, (p. 31) 

Second, if "one wishes to discover a minimum or maxinium ... a 
biased selection may be preferable to a broader coverage by a prob- 
ability sample” (p. 32). Finally, « «Mn« exploratojy studies such 
samples are iJie only kind one can take tlic time or trouble to use. 

In general, then, judgment sampling should be restricted to situ- 
aUons in which (1) tlio possible errors are not serious and (2) prob- 
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ability san.plin, it practically impale Data from 

pics at best only saggest or parameter 

a sound basis either lor obtaimne stat.sl.eal “tmates »' P 

values or lor conducting statistical tests such as will be consi 

the next two chapters. 


OPTIMIZING SAMPLE SIZE 

As indicated earlier, the too methodological P™'’'®™ “‘mplt 
with sampling involve dctenniaaUon oi the sise “ JP ^ 
to he used, first we consider the problem o tipped 

shall see, a decision model of this research problem can be eonstra 
and solved under certain condiUona Hiscus- 

In the literature on sampling theory there .5 a involved 

Sion of optimising sample site, but the typo of “P*’® , . ? j [or a 
is usually very limited. Sample s.so .s nomally 
specified sampling design and a fixed (^ce.ficd) ® . jonjple 

verscly, sampling error is miD.m.zcd .a some cases p 

site; for example, in optimal allocation of the sample to ..mple 
228). Our objective here is U. convert minimised, 

size into commensurable costa the som of wh.ch .3 to [ 1,0 

In order to accomplish tills objective it is necessary P 
cost ol error as a luncUon of iU magnitude. . 

First ve will consider dclcrtninalion of optimal sample s 
the cost of error is assumed to be a quadratic function of i 
tude; that is, under the assumption that the cost of error is g 

C(p - V) - Uy - y?, 

where C{.y - Y) = the cost of the error involved in using a 
for the value of u parameter Y. 
k ^ a cost constant. 

Note that in tliis assumption the cost of an error (1) is the 

tlic true value of Y in that it depends only on the magniturte ^ 
position) of the error on the error scale, and (2) does not ep 
the sign of the error. re^ultiDS 

The quadratic assumption is generally made because of tnc 

mathematical simplicity and not because of conformity 
real situation. After discussing how to optimize sample size u 
assumption, wo will consider how cost-of-error functions 
rived from a decision model and used in this optimization pro 


u estimate y 
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Simple Bandoit) Sampling, Quadratic Cost of Error * 

3n tljia and the succeeding cases ia trliicli quadratic costs of error 
are assumed, we will also assume that »c arc trying to estimate the 
mean of a normal (or aj)proxinialeIy normal) population. In the case 
of simple random eampling the cost of Uic sampling is assumed to be 
given by 

C(n) = C, d- nCa, (2) 

nhere C(n) » tlm total cost of a obscrr-alions. 

Cl = the fixed cost of preparing the sample. 

Ca = Uie cost per observation. 

Let be an unbiased estimate of tlie variance of the {)opuIation. 
Then 5*/n is tlic variance (mean squared error) of estimates based 
on samples of also n. From Ibe assumption that the cost of error 
is proporticinal to the square of the error the estimated expected cost 
of error, B(C), is given by 

Em-i— ( 3 ) 


If the true variance, c^, is knonn (in this and succeeding eases), it 
can bo substituted for tJicn the expected cost of error would be 
known ratlier than estimated. 

The total expected cost, TEC, of a simple random sample of size n 
which is designed to csllmalc Ibo mean of the population is Uicn 

5 ^ 

TEC -=Ci + nCa + k— (4) 

» 


To find the optimum sample size, woprocceil as follows: 


dTEC 

dn 



(5) 


This can be shown to bo .a jainirowtn point. Then 



• The juateml ia Uiis ewUon aaJ U»c oihcre »iilch iJtal wUh auaJfatic co»U of 
error U due to mr coUea£ue, Rudolph Reinitz, 
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Therefore, the niirritituin total c«pected cost is given by 

& 

TEC. = C, + Cj 

= Cl + IsVkCt. 

Strolilied Random Sampling, Quadratic Costs of Error 

The population of N units is divided into L erelusive and eJiaoBtive 
strata so that /gt 

Wl + Wa + -"W»+'"+®t“ff. 

Here the problem is to select the sample sire ns ‘'O™ f S 

ivill minimize the total expected cost of that sample. The total T 

cost of the total sample, n - tn., is minimized if each stratum cos 

i, minimized. Ncyman (1934) demonstrated tota‘ 

the variance of the estimated mean, S, , is smallest ^ „ 

sample size if the sample is allocated frith ns proportional to JifsSi, 

Ss is the standard deviation of the hlh stratum. That is, it 

WlSs (9) 

"‘““jATsSs 

wo obtain yj) 

n ">■— g ~' 

NkSk 

The expression for the minimum vaxiaoco is 

Assume that the cost of sampling is given by 

C(n») = Cl + SnftC*, 

where Cs = Uio cost per observation iu the hth stratum. The estiin. 

expected cost of error is obtained from equations (9) and ( )■ 

E(C) = k min (S,*) 


\nx / 


( 13 ) 



Therefore, the total expected cost is 
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To get the 


TFC.C,-t-SJ„C, + ^S(~-Ni)s,\ 04) 


optimal sample ase for stratum A, we proceed as follows: 


t>»* * «** 


05) 



(IC) 


The mimmal total expected cost of the sample is obtained by substitut- 
ing the value of tia. from equation (Ifl) in equation (14). 


Cluster Sampling, Quadratic Costs of &Tor 

In euster eampluig the cost of preparing and obtaiitiog the sample 
may require a very eompicY expression for it« description. (See, for 
mmple, Hansen, Hurwitz, and hfadow <1953, Chapter fi, Section 
D).] In the simple case this cost may be expressed as 

' C(m,«)-m(C| + nCi), (17) 

vhere C(m, il) ■> the total cost of preparing and obtaining the sample. 

Cl •=> the fixed cost of preparing n cluster. 

Ca *» the cost per observation. 

m » the number of clusters in the sample. 

fl = the average number of obacrvatimis per cluster. 

llio problem is Co fioJ the optimal xslucs of nt and A, and He, rei.pec' 
tively. 

The Unbiased estimate of the XTiriance of the mean of die population is 
given by 

« hero a1f =» total number of clusters in tlie p«>pHjation. 

N => total number of elements in the population. 

5a® = estimated variance iclativo to y bctuwn clusters. 

5|,® ** estimated variance relative Co y within clusters. 
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The total expected cost is then pito by 

T£C « kV(i/) -h C(m, fi) 


where fc b the a-nia! cost constant. Forfimplification purposes, since fl 
appears In (19) only in the expression mfl, let <2 =* Thco, rewriting 
(19), we get 

= ^ [(^ - h) + G - Jfs) *’’] + 


To find m« we proceed as follows: 


— j- + Cl *= 0 


Mow, let us find the optimal value of Q, Q,: 




_ (?> S, Vt/Ca 

m, 5 > Vk/Ci 


By appropriate substitution of the expre^oos for m, and Q» in equation 
(3)) the minimal total expected cost can be found. 


Nonquadratic Costs of Error 

Now we attempt » general formulation of a model for optinuzuii 
sample size, in which the costa of error may take any form. 
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t«l r rrprcuiiit tlic true average value of a population characteristic, 
and let r rcj)resctit an c.stimato of Uiis mean. Then, if ;? > P, there is 
an error of ovcrestiinatiojj, imU if P > j?, Uierc is an error of undcroilj- 
matiun. Ixt n'prcient Uw rtfcl of an error of underestimation, and 
C, tho cost of an error of cn-ettabioaliou. T3<e values of aad Ca mil 
gcjjcrejly depend on tJjc sixe of tbo error. 'I'hat «, /?„ is generally a func- 
tion (s) of (P — 0, nhcrc T > §. 

C\ » pfP - ii), P > jj. (20) 

Siiitilarly, 

C, » MS - P), S> P- (27) 

Then the cxt>eclGd co^t of uudcrcstimaliuu is 

X r 

s{V ~ DptBilii, (2S) 

uhere p(S) is (he prabahility-dcitsity funrtion of the estimate, Simi- 
laHy, the expected oust of ovcfestimation is 

£ to -!’>?&) Ji. m 

Tlio bUm of Ihcoo tivo expected cohts is tho total expected cost of error. 
The cost of preparing the samplo and ukingnobscrvnliuns is given by 

Cin) « C, (30) 

We u<=c Uiis form to aliovv for Uw po.sdhility that the co^t of taking n 
ubserx’utions may not Iw sv linear function of n. 

Tim total expected cost, that, is gi'xn hy 

X F 

g{7 -v)p(8)<l3 

?)p(i)#. (31) 

To iliuslrjjtc the u^j of Uus cqualion consider tljo following simple 
case. rUsumc tJiat tJie costs of OTcr- and underestimation are a linear 
function of tho rnagniludo of error. (Wo shall have more to say on tliU 
Sunctioo helm-.) Thro, Jetting h equal a unit cost of undorcstimaUon 
and ki a unit cast of overcstinwUon: 

cc? - 0) - j(r - j) - wr - s). ?> s 

and „ 

C(!/ - P) = M9 - P) =* fca(S - n, 5 > P. 


(32) 

(33) 
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We shall again assume that 

cw - C, + nC,. 

Then aesnnhns Ihnl the ^-thna.cn arc nnnn-nlly .Wribated about .te 
tnio value, V, equation (31) tan be teanUen aa 


TEC = C, + nCi + J ti(? - S) 

~ r „ 
+ Jr 


j—u-ri’/a-'Jj. (3o) 


Tills can be simplified by integration as foilons: 



» Cl + nCj + 


0.49(1*1 + fcj) 

V^i 


To find Uie optimal sample size, n,, we talto the dcrivativ e, set it 
to zero, and solve for n:* 


dTEC „ 0.29(1:, + 1 : 2 ) ,, (31) 

V? 


This value is m inim um because 


Then 

and 


- > 0, for all n > 0. 


0.01«*(fci + 4 * 2 )* 


^ p.IHr’Cbi + 


• n of course, sliould be an integer, but here v.e are treabDg it 
.ariable anil are rounding oS Uie solution to the nearest inUger 


(3S) 

(39) 


( 40 ) 
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Consider tlio following itumerical example: 

i^i » $050 
ka = $0l25 
« = 4100 
Ca = $0.50. 


Then 


«« 


(Q.01)(nOQ)^(0.50 + 0.25)^ 

(oTso)* 


’ 1,512,000 


n* = 115. 

Determining tlic Cost of Error from a Decision Model 

In applied problems, the value of tbc "cost” of error can be deter- 
mlaed Srota the decision model in tenns of loss of performaoce. For 
example, consider the following simple decision model: 


(41) 


uhcro V la a nioasuro of perfornninco to be rolDinuscd (o.$., cost), .Y is 
the control variable, and 1* Is the Forameler whose value must be deter* 
mined. To find the value of .Y wbicli mlaimucs F we proceed as folloua: 


Then the minimum v.aluc of F U 

y . 


dV 

I'-i-O 

m 

A', =» 

1 

Vy' 

m 


Vy 


Vr«2v^. 


(44) 


Kow, if an estimate y is used for Y, then instead of A'* wo get 


and instead of V, u e get 




'Vi' 

+ Vi. 


( 40 ) 
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The cost of the error, y ~Y, is 

C(v - y) = V. - 1'. ' 


Vi 


+ U 


-iVy. 


(17) 


It trill he ohscrved that in this cspressron for to cost of er 
the true value of to parameter. Y, appears. Hence, in 
the cost of error is not only a function of its ”;»SU>tu J= J y 

a function of the location of this magntode on the 
is measured. Furtliennore, this cost-of-error function 
metrical; it depends on the agn of the error. This w 

proven inerally in to nert chapter. It can he ■ ^ 

assuming Y = 4; then for the indicated values of y the following 
of error are obtained; 


Cost o/ iVor 

4i 


li 

n 

3f 


As shown, the cost of a particular error of estimation ^^5 

illustrative model wc liavc been using depends on 
the parameter being estimated, V, and the estimate of t a p 
y. This is true for the cost of esUinating errors '^ssociateO w 

parameters in most decision models. Clearly, if we knew 

there would be no need to sample and estimate, and » J 
would already liavc an estimate. The need to know the vaue 
estimate y can be removed by a knowledge of the distnbu 
estimates around the true value, pfvjy), which is normally P 
by the theory on which the estimating procedure is based, -c -v,dod 
distribution allows us to deal with the expected cost of error pr 

wo know the value of y. Siimlarly, if we know tlie distribution 

true value of y,p(y), or can make some reasonable f ^ 
oivmif w «o rnn imp Ibis distribution 33 well. That IS, the e p 


about it, we can use this distribution as well. That i 
cost of error E(C) is ^ven by 


£(0 “ y)p(yiy)pcndydy. 


(48) 
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\%here R and r represent the range of possi!>Ie values of I' and y, respec- 
tively. The total evpertcd cost (obsenations plus error) is 

TfiC = C(n) + E(C). (19) 

Then, in the example, 

rsc - c, + «c, - np<!/l npd') dijdr m 

In nclual practice the distribution of the true value of the parameter 
can sometimea be estimated. For cxain])lc, Y may rcjwcsent tlie 
demand for a product in a future month. Actual past monllily de- 
mands arc knoum. The average past monthly demand may not be 
used aa a forecast for future demands because a better forecast, y, 
can bo prepared based on information obtained from a sample of po- 
tential customers. 

But even whore pf}') is known the task of perfomuDg llic neces- 
sary mathematical operations to find n* may bo prohibitively complex, 
if at all possible An appro.ximatcIy opUmal sample size can be ob- 
tained, however, if tlie following procedure is followed. 

(1) First rewrite (50) in a conditional form 

(TEOl !■) - Cl + nC, + Jcto - npCsl 1') *, (51) 

where the values of y fall witliin the range 

r = 1(1' + Ko) — (F — Ko)l stoiidard units. 

(2) Since the integral involved in (SI) is over a rettricted range, 
it must be normalued. Po Ihia .vs follows: 

^r-trr 

J C(y- l>(yjF)dl/ 

(reel )’) - Cl + »c, + ' 

' f p(yj y) dy 

(3) Assume a value for Y and solve for n, by numerical integration. 

(4) Continue for a variety of dUferent values of F. 

(5) From the results obtained fit a function: 

n.=tf(0. (53) 

(G) On the basis of the best- mfonnalion available (o.g., from 
previous study or a prolimiuaiy sample) get an estimate y, and aa- 
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sume Y = y. Then, using tlic function obtained in step (5), select the 
corresponding 

If observations can be taken eequcntially and estimates of Y pre- 
pared after every one or few observations, tiicn ng can be determined 
for each estimate obtained. As soon os no n, is obtained which is less 
than or equal to the sample already drawn, the sampling is discon- 
tinued. For example, let 

Ci “ O.OOJ 






[- 


(II - n^n ] 

2s’ j 


Solving equation (52) and following steps (3) tiirough (5), we obtain 
the following values of n, lor specific values of Y : 

Y n. 

G 28 

8 21 

10 18 

12 IG 

14 14 

The plot of tlicsc data is shown in Figure 7.1. 


T i — ^ — I — ! — r 



J ^ I- 1 1 1 1 

6 a iO 12 14 16 IB 


FioURS 7.1. 


_ J{y), cijuatiOD (63) 
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OBSERVATIONAL AND SAMPLING ERROR 

Up to thi. point tho ciiscuB^on of optimizing enmple size has not 
taken observational error into account. Observational error it ;v 11 
be recalled from Chapter G, may be due to tlic lehaMor of the ob- 
tlTTe oLrvcd. Uic instruments used in obsers-mg, and/or tb 
environment in whith Urn observations arc made. Now wo shall 
consider such error. w „ 

consider a 

particular proper y ^1 aaribnlcd with a standard dcviaUon, 

„1 thoekmon^, yo”'™S.s n^o on the ttl, elcrnent an= normally 

Assume also that oWahom n^ The standard do- 

n clomcnta e.-mctly once, has a standard error «bere 

( 54 ) 




dSu“enofobser«.io^|;^^^ 

The value of v, as pv en in to ' ^ enamplo, lor simple ran- 

" 3'£0-C, + nPa + *(v +'“’)■ 

, 1 w.-«tinnal CTfor lu Chapter G indicated, this 

mrer“LtSra" "" 

That is, 




- <KCa)‘ 


(50) 


It becomes clear “"'T ^rTgmmaT opUmiraUoa piob- 

which minimirc the total espeded i»st. 
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ContJDujng will) Ujc case of ample random sampling witli a quad- 
ratic cost-of-error function, equation (55) becomes 

TEC = C, + nC, + — + (57) 

n 

To find tlie optimum values of n and Ca we take tbc partial derivatives 
of TEC with respect to n and Cj, set each equal to zero, and solve 
the pair of simultaneous equations. For example, let 


Hi = 0(Cj) = 


where iv ia a constant. Then equation (57) becomes 

TBO = C, + n(7j + — + 

n Ci 


Proceeding aa indicated, 


dTEC ^ 

-T- - C, - » 0 

n* 

6TBC 

iC, ““ 


Solving (01), we obtain 




(58) 


m 


(CO) 

(Cl) 


( 02 ) 


Svibstituling (62) in (GO), we get 


\ n tr 

(63) 

which reduces to 



(04) 


Substituting (61) in (02), we get 


(65) 
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The procedure described can be used where <71 aad <73 are cot known 
but aro estimated, by substituting the estimates /or the true values.* 


THE RELEVANCE OF TIME TO SAMPLING 
AND OBSERVATIONAL ERROR 

Failure to cousider the elTect of time on observations can lead to 
significant sampling and observational errors. First let us consider 
the relevance of time to sampling a population of elements. 

In principle, the parameter of a population to be estimated, V, is a 
property of Uiat population at a specific time. For example, the 
average height of American men and the number of people living in 
the United States are properties which vary o^’er time. Consequently, 
the parameter to be estimated sliould have a time Icdicator associated 
with itj for example, y». To estimate Y, we take a sample of n items 
and observe each, hut these obsers'aUons are not all made at time t, 
AS they be ideally. Ilrey should, then, ho adjusted to what 

would have been observed at time t. 

Necessity /or adjusting obscr\-atioQS to t is removed by the assump* 
tion that Yt does cot depend on t. This may mean either of two 
things: ( 1 ) Vi may be regarded as icdcpe&dcot of t if and only if 
Y, is constant for all t; or ( 2 ) Y, is a random variable with a distribu* 
tion fuocUoD that does not depend on t. If Yt is assumed to be con* 
Btant for all t within the relevant interval of time, then the time of 
observation is irrelevant. U, however, Y, is assumed to be a random 
variable with reject to i, Ihen the time of the observailons is im- 
portant. 

In some cases, when Uic variation of Y with respect to t is small and 
random, it is supposed that Uic property being estimated is the mean 
value of y over an interval from /• to ti, y#i. To esUmale this param- 
eter properly the time interval should also be sampled. This is 
seldom done. Fatlisr, occ usually acts as tliough the observations 
were all made at t. Whether or not it ia reasonable to do so depends 
on how much variance of observations is due to variations in t This 
time-related variance may increase with increases in eample size, 
perhaps more rapidly tlian sampling error ia decreased. 

Consider now the case 0/ a sin^e object whose property Y is to 
be estimated. Here too the lime subscript ia rdevant if y ia not 

*Mar£chsk (JOSS) and lUdoer (IflSS and 1B50) havo also cao^deml tiie cost 
of transmitting the observatioDs to a central Leadquarlers or to othen in the 
field for dccisoa-makiog purposes. 
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fixed for all C in the period of interest. At l/est on)y one obseA’alioD 
can be made at time 1. If I is relevant, then either (a) controls 
must be imposed to liolJ )' constant (as in a laboratory), or (b) 
adjustments in the observations aro required, or (c) time-produced 
variance should bo considered in determining tlio number of obser- 
vations to bo made. 

When the parameter being estimated is a random variable with 
respect to lime, it can bo treated as an observational error. This 
type of ob 5 cr.'atioiial error does not necessarily decrease as the time 
per observation is increased. 

.Although in principle the above remarks are quite obvious, in 
practice they are frequently ignored with consequences whoso sig- 
nificance is left undeterrained. 


CONCLUSION 

In order to construct a decision model of the selection of a sample 
design it is necessary (a) to be able to express the differences between 
sample designs in terms of quantifiable variables, and (6) to express 
errors of esUmates yielded by these designs as functions of these same 
variables. In order to accomplish Uic latter it is further necessary^ to 
know a great deal about the distribution of the property in question 
among the elements and in tbe subgroups of the population to be 
sampled. As yet no such general decision model has been constructed. 
More specialized models addressed to less general desigp questions, 
however, ha^’e been constructed. With the development of more and 
more of these specialized models, wo gradually approach a general 
sampling-design model. 

In order to select the best of a set of alternative sample designs, 
it is necessary at present lo deterroiae (or approximate) the optimal 
sample size for each design, and then select that design which has the 
least total expected cost. Such an iterative procedure is the only 
one available to us until a feasible way is developed for representing 
differences in sampling design by values along one or more scales. 

Determination of optimal sample size for any particular sample 
design requires a knowledge of the cost-of-error function of the vari- 
able being estimated. As we showed, Uiis function can be derived 
from the decision model. Since this cost is generally a function of 
the true value of the parameter bong estimated, it is necessary to 
use approximation procedures. The consequences of the frequently 
made assumption that the cost of error is of a particular form (usu- 
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ally quadratic), which is independent of the true value of the param- 
eter being estimated, should be carefully weighed before taking 
advantage of the ninthemaUcal sioiplicity wlilcli &ucli an assumption 
usually yields. 

In "pure” science, whcic decision models cannot be coiisUuctcd 
some assumptions concerning Uic rost-of-error function are necess.ary. 
These assumptions should be made capticit, anti sufficient data should 
be provided so that users of the results cajj adjust liiem appropriately 
for the content in ivhich ihey are to be used. 
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ESTIMATION* 


INTRQDVCTIOS 

The paratBCCcfs in a decision model, Y), represent either the property 
of a single tiling (object, event, or state) or the property of a coUec* 
Uott (population) 0 / these tbtage. Multiple olscr>*ationa of a gingio 
thing may vary for either or both of tn-o reasons; (1) obser^'ational 
error, oad/or (2) temporal error. The latter error is introduced by the 
fact that ohsenations cannot be mode simultaneously and the prop- 
erty being observed may ehongo over the period 0 / observation. la 
the e.'ise of a collection of things, the olKer>’ation3 may vary not 
only far the tiro reasons cited but also because of variation betweco 
U;e properties of tlie items being observed. 

If a scientific obserser finds 00 vanatioa among bis observations, 
bo acccjils tljis not as Ujc attainment of perfection, but raUicr as 
an indication of a serious imperfection. lie may suspect Ibat his 
measuring instrument is januued or that he Is not making bis rcadbgs 
precisely enough. Ilo investigates to find the cause of eonfonnity 
and "corrects" it so that he gels variation among obsen'ations. This 
process yields cvcr-incrc.'ising accuracy of observation. 

* Tlio malrnol in this ilisptcr is bunt hiseir on research at Case Institute of 
Teehaotegy, sponsorej by iSio OiSee of tiini ^csearcfi, Navy Contract Ka. 
NONIl-IUKOS). Most of the work on tW# ptoiect was performed by Dr. Shiv 
IC. Gupta (1900) ivnd Professor Frist tlaiusmaiiD. Portioas of their various wnle- 
upa of thU work have been incorpomted inlo this chapter with their permision. 

251 
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Given a set of numerical obscrvaUons of a properly Y {yi, yi, 
•••» !/•)» what particular number should be used to represent the 
property involved? This is the problem of estimation. Put another 
way, the problem ia to find a number y, where 

y = '".y-) 

such that this number is the “best" estimate of Y, To solve this 
problem we obviously require one or more criteria of “best" in this 
context. Statistical estimation theory has been concerned with de- 
veloping estimating procedures and criteria for evaluating them. 


SOME IMPORTANT ESTIMATINC 

proceduhes in statistics 

Three major (but by no means the only) estimating procedures are: 

(1) The analogue method or method of moments. 

(2) The manmum-likelihood method. 

(3) Bayes strategies. 


The Analogue Method or hfethod of Moments 


This method, which was developed by Karl Pearson, may be desenbed 
loosely as a procedure of estimating a parameter or property of a prob- 
ability distribution by the same property of the sample. For example, 
the mean F of a finite populatioo is 
1 ^ 

— TYi 

jvr ' 

where iV is the number of dements in the population; its analogue esti- 
mate is < « 

e = _ V V.- 


where n is the number of elements in the sample. More precisely, using 
the method of moments, we estimate a parameter Yh, which is equivalent 
to the expected value of the fclh moment * of P, by yt, where 


* In general, a moment is tlui mean Tslue of a power of a variate. Tberefore, t2ie 
tUi moment of a variate T with a cumulative distribution F(}') is 


J* r‘dF(io, 

and its moment about a particularvatiiea n 

(P-apdF(r). 

The first moment about the U zen^ and the second moment U tbe vanance 
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n--rr.‘, 

n I 

ivliich ia monjCHtof thcAampIu. Similitriy (ho analogue eslimalo 
of Uie variance of i*. o)'*, whicli is cquivaient to the expected value of 
(F - D*. U 

*»■* - E (ff. - sf. 

n j 

The ineilian of a sample is au analogue estimate of the median of Uie 
{iopufation, and nti ulk^HrrviHt jiroiKirtion is an analogue estimate of a 
probability. 


The ^faxuI)ut^•L^I.elifloocl Afcthotl 

Let V. Tcpa'scnt a particular random sample of size n which has 
been drawn for the puriiosc of cslimaUng the value of the poputation 
[lanitaotcr Tiicn the tikcliiiood of a particuiar value of F(}'i, Yt, 
* ' •) 13 defined os the conditional probability of drawing the particU' 
lar sample Ft, given F; that U, plF*)F). The maxiioumdikdihood 
method consists of o>timaUng (lio value of F by that one of its pos* 
sible values which luaumizcs the likelihood function. 

For example, su])pu»c tlhit F can take on three possible values, 
Fj, Fj, F|, sad euppofO that there arc three possible samples (pi, y*. 
and j/j) with probability-density function p(j/ijl/>) as shown in Table 
8.1. U is clear Dial, if the sample ]/i is obtained, the likelihood 
?(VilF«) is maximum for F|. Similarly, F3 and F| yidd maxima 
likelihood for y* and y,, rfeiK;cti\-cIy. Jlcncc, tlio values of i'l, Fa, 
and F* are tnaximuin-lifccJihood t-Uaiates corresponding to the sam- 
ples y], Vs, and J/j. 


Taiux 8.1 



yi 

vs 

V> 

Fi 

0.70 

o.:20 

0.10 

F, 

0.30 

0.50 

0.20 

"I 

a.{s 

•32S 

<J.03 
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Nonaally, of course, Y can have infinitely possible values, but the 
same lo^c applies. 

In some cases * the sample mean y is a maxunum-Iikelihood estimate 
of F. The obsen-e<] ratio of m ouleomes of a particifiar t^pe to n trials 
(le., m/n) is a maximuia-likeUhood estimate of the probability of that 
particular of outcome. 

Bayes Strate^es 

Suppose that on some grounds wc have an a priori distribution of 
possible values of Y, p'fY). Then the sample obtained can be used 
to revise these so as to obtain an a posleriori probability distribution 
p(y). In EO doing, the true value of F is regarded as a random van- 
able, and hence the cost of using an estimate y when the true value w 
F, C(Y, y) may be regarded as a random quantity with a distribu- 
tion which depends upon pfV). The value of y which minimizes tbe 
expected cost of error, E[CiY, y)J, where the expectation U bared 
on the a poUeriori distribution, p{Y), may be called a Boyei 
mate. 

For ejcample,t suppose that there are two possible values of Y, Fi 
and Ys, and three possible seta of observations (yj, yst a^d yi) with 
the conditional a priori probabilities shown in Table 8.2. 

Then the (unconditional) o prion probability of obtaining each 
estimate, pf{.Y)p{,yi\Y), can be computed as is done in Table 83. 

The a poaferiori probability for each possible value of Y and for 
each possible sample U determined as shown in Table 8.4. 

5fow suppose that the costs of error are as shown in Table 8.0. 
Then the expected costs of error can be computed, and for each pos- 
sible observed stale we can find the Bayes estimate; that is, the 
that mimmizes the expected cost of error. The computations are 
shown in Table 8.6. 

Up to this point we have considered only the case in which a de- 
cision depends on the value of one parameter. Jfow consider the care 
in which the decision depends on estimates of two unconnected param- 
eters, Y and Z. To illustrate the lo^c of the Bayes estimate in this 
case, assume that F and Z can each take on only two values, Fi and 

• For these esses, see J. M. Keynes, “The Fnscipsl Averages and the Laws 
of Kiror Which Lead to Them,’* /pumol «/ iAe Rogai Siaiuiieo] SodelVi 
(19U). p. 322. 

tThis example is adepted fivto CbeiaoS tad Hoses (1SS9. pp- 2S0-2St) 
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Tasle 8.2. CasomoXAh A Priori PBOBABitmcs 



yt 

V* 

yt 

p'(y,) 

I’l ' 

aco 

0.23 

0.15 

o.co 

Y, 1 

020 

030 

0.50 

0.10 


TAJitE S.3. UNCOKomoSAL A Priori PROBiBiuriEs, }>[]/,) 




yt 

yt 


Q.G(O.CO) » 0.30 

0.6(0.25) - 0.15 

0.8(0.15) - 0.00 

>*t ! 

0.4(0.20) - O.OS 

0.4(0.30) - 0.12 

0.4(0.50) - 0.20 

sp'cnp(i/.in - p(y.) j 

a44 

0.27 

0.20 


Tabi-b 8.1. A Poittriori P»oSAitiUTics 



I vt 

Vt 

Vi 



0.15 


\ 0.44 

0.27 

6.20 




0.20 

j 0.14 

0.27 

0.20 


Table 8.3. Costs or Error 
Yt Yi 


Yi 
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If these probabilities arc not independent, appropriate adjusUaents 
can be made. 

The Eubsequent Eteps in this case now correspond to those shown in 
Tables 8.3 to S.O. 


STATISTICAL CIUTERIA OF “GOOD" ESTIMATING PnOCEDUKES 

The criteria most frequently used to evalutttc estimating procedures 
aic: 

Lack of bias. 

(2) Invariance under (ransfomution of parameters. 

(3) Consistency. 

(4) JSScicQcy. 

(5) Sufilcicncy. 

Wc want to efedno each of Uicsc criteria and to evaluate the esti- 
mating procedures uith respect (o tfieoi. 

Lack of Bias 

An estimate y oi A parameter wbosc true value is Y is said to be 
unbiased if iU expected value, F(y),t is equal to Y. Hence, the 
measure of hiaa is E(y) — 1'. For an unbiased estimator, B(y) - Y 
^ 0 . 

For example, y = ~2v» is an unbiased estimate of |P; sj-* = 
— - — S (y< — y)* is an unbiased eslimafe of ar*. On the other hand, 

rt — 1 j 

it can be shown that is a biased estimate of F^. 

Invariance under Transformatioa of Parameters 

An estimating procedure which yields y as an estimate of Y is 
invariant under Irons/omialton of parameters if it estimates some 
transformation of Y, F* =» fffF) by y* = gfy). For example, let Y 
be the Icng;tli of an object in inches and F* be its length in centi- 
meters. Then F* = 2.5401’. The "invariance’' condition asserts that 

tTbe expected value of v, *• dw bmitins value of the average of the 
estimates obtaiocil as the buoifrer of aompteS of the mme sue approaches is- 
finity. 
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if we estimate Y by ^ we siN>u](l csUmate Y* by 2.540y. Tlie maxi- 
mum-likeUhood and (if tlie cost function and prior distributions are 
invariant) Bayes estimators satisfy this condition, but in some cases 
the analogue method docs not. 

Tlic criteria of unbiasedness and invariance may not both bo simulta- 
neously satisfied. For example, attbough $ is an unbiased estimate of 
F, j;* is a biased estimate of F*. The criterion of invariance is generally 
taken to be the more fundamental. 

Consistency 

An estimating procedure is said to be consistent if the estimates 
which it yields tend to approach the true value more and more closely 
as the sample size is indefinitely increased. Wore precisely, the prob- 
ability that the estimates converge on the true value must approach 
l.Q as the sample size approaches infinity: 

p(y F) -» 1.0 as n se.t 

A consistent estimator is almost always unbiased in the limit (>-e.| 
when ntB !7, the population size), but for any sample size less than 
that of the population (i.e., n < N) it may yield biased estimates. 

All three methods of estimation which we have considered yield 
consistent estimates of separately estimated parameters. 

Efficiency 

An estimator is said to be asymptotically efficient if the distribution 
of estimates which it yields tends to nonnaUty with the least possible 
standard error (i.e., standard deviation of the estimate) as the sample 
size is increased. Therefore, Uje standard error of estimates yielded 
by an efficient estimator must be no greater than that yielded by any 
other consistent estimator which approaches normality in the limit 
An efficient estimator must also be consistent, but it may yield biased 
estimates where n < iV. 

The analogue method docs not always yield efficient estimators, 
but the maximum-likelihood and Bayes methods generally do. In fact, 
if we start with any probability-density function of Y, which has 
positive density for all valnes of Y, the maximum-likelihood esti- 
mator and the Bayes estimator (assuming a smooth cost-of-error 
function) will be extremely close to one another for large samples. 
tThe symbol represenU "approaohea the value of.” 
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Suillciciicy 

An estimating procedure yieWs lUiJIciVnt estimates if no other sta- 
tistic can bo derived from the sample wliich yields any additional 
information concemine tJjc parameter being estimated. Afore pre- 
cisely, it y,, 1 / 3 , • • y, is a sample from a j>opulatiDn with density 
p(j/jy) and if yj, •••, y,> U such that tlic conditional distri- 
bution of Vi, y3, •••, y« given <|^ docs not depend on V, tlicn ^ is a 
sufTicicnt estimator. Such c&tiin.ator8 arc generally considered to be 
tlio most desirable but Uiey exist only in rather special eases. 

A eutliclcnt estimator is usually etScient and consistent. It can be 
s(io;m) ti]:it, if there arc sutGcient estimators of a parameter V, the 
ni.iximtim-ltkeliliood method uiti yicM otic. 

nevicH'ing tlic litrce {neUmds brietty, Uic analogue metiiod is tisu- 
nlly the cnsiret to apply, h requires the Ic.'ist information; it docs 
not need complcUi knowU'dgo of t)ie probability-density function. 
AlUiough it always yields consi$U.-nt estimates, it docs sot generally 
yield ctScient ones, noth the maximum-likelihood and Bayes meth- 
od-s rettuire knoalcdgo of p(v]l'). Doth generally yield clTicicnt esU- 
malcs \^hich are invariant under transfonnntions of parameters. If 
aulCeient cstioialora arc possible, the maximum-likelihood method will 
yield one. The Bayes method additionally requires knowledge of the 
probability distribution pfl’) and U»« cost-of-error function CfK, y). 
liccauMi of the complexity of the Bayes method fuhtch we will con- 
sider in more detail below), It has been used primarily ss a thcorct- 
ic.'il tool, uherciis tlic otlicr tuo methods itave had wide practical 
application. 


PRAGMATIC CIUTERION TOR BEST ESTIMATES 

An estimating procedure which yields unbiasat estimates wiui 
minimum variance will yield the minimum expected cost of (esU- 
niating) error only if Uie cost of error is a quadraljo function of its 
magnitude; that is, if 

Assuming that the cost of error is quadratic, a biased estimate with a 
8m.all I'ariaflce way }«> preferable to an unbiased estimate with a 
large variance. For Uiis reason Hansen, Hurvvil*, and Madow (1953) 
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suggest use of the mean square error, which they define as “the vari- 
ance of the estimate plus the square of the biaa” (p. 56). An esti- 
mator which minimised mean squate error would minimise the ex- 
pected cost of error whether it yi^cd biased estimates or not, but only 
ij the cost-of-error function is quadratic. The quadratic function is 
normally assumed for three major reasons. 

First, many snooth [cost-of-eijor) functions which vani^ at [y = F] are 
well approximated by the “squared loss’’ function, especially for [y close 
to F]. Second, the mathematics involved in using this . . . function is r^- 
lively simple compared to other . . . functions. Third, for large Mmples, 
many reasonable (stimalors yield estimates whose probability distributions 
are approximated by normal distributions with mean F. Such distributions 
are completely specified by the variance of F, and the smaller the variance 
the better the estimator, no matter what the [cost-of-error) function may 
he (bo long as [C(l', v)l increases as y moves away from F). [Chemon 
and Moses (1950, p. 276)] 

The second reason, mathematical simplicity, is generally valid; but 
the other two reasons arc not The first is largely a matter of wishful 
thinking. Most cost-of-crror functions generated from a decision 
model (as we shall do below) arc not well approximated by a quadratic 
form because, among other reasons, they are very seldom symmetrical. 
As for the third reason, it is true that "the smaller the variance the 
better the esrimator" no matter what the cost-of-error function may 
be (under the stated conditions) ; but the estimator with the smallest 
variance Is best for only certain types of cost-of-error functions. 

From any particular decision model one can extract the cosl-of- 
error function for a specified parameter and hence can also determine 
whether it is symmetrical. For example, consider again the very 
simple decision model used in Chapter 7: 

K-/(.V, 11-.YK + |. (1) 

where V = total cost, -V is the control variable, and F is the param- 
eter. To find the optimizing value of X we differentiate with respect 
to A”, equate the result to zero, solve for X, and obtain 


X, 


P(F) = 


Vy 


The minimum cost, therefore, is 

V. “ /Isd'): n - A'.F + Y = 


( 2 ) 


(3) 
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In normal scientific practice an estimate (t;) of V is obtained and 
substituted for Y in equation (2), which yields an estimated optimiz- 
ing value of X‘. 


=‘9{H) = 


V? 


(4) 


and the following minimum cost is obtained: 


Tiius, tlie difference between tijc estimated and the actual minimum 
cost, the cost of error, is 


“ (0) 

As indicated in Clmpter 7, tJ»e cost of error for this decision model 
depends upon the sign of (y — V) = aV, the error made in esti- 
mating y, and hence is not eyiumctrieally distributed. This is true 
because otlienvise wo must have 

^,=^+Vr + i,--2V? 

- y,/ ^y + Vr-Ar-2v1^, 

which is true only if aK = 0. 

If the cost of error is not symmetrically distributed about the true 
value being estimated, and if this cost increases with the magnitude 
of t})e error, tiicn an unbiased estimate — whatever its other properties 
—will not minimize the expected cost of error. Furthermore, an esti- 
mate which minimizes variance will minimize the expected cost of 
error only if this cost has a quadratic loan. The advantage of the 
Bayes approach, in principle, derives from the fact that it can yield 
estimators which minimize ibis expected cost whatever the cost-of- 
error function. 

After reformulating the problem of esUinaUoa in very general terms, 
wo will consider two specific procedures for generating estimates. The 
first procedure obtains a Bayes estimate by use of the calculus of 
variations, but it is much simpler than many of the procedures avail- 
able for obtaining such estimates. The second procedure, the bfering 
metliod, unlike the variational method, assiunes the form of the esti- 
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mating function but it does not use the a priori distribution of the 
parameter being estimated. However, it does make some assump- 
tions about the parameter. 


^tATHE^UTlCAL FORMULATION OF 
TUB ESTIMATION PROBLEM 

Detennination of the expected cost of (estimating) error requires 
knowledge of 

(1) either the true value of Ose parameter, Y, or its probability- 
density function, p{Y)-, 

(2) the probability-density function of the estimates, p(yjy)» 

(3) the cost-of-error function. 

If y is knonm, of course, it need not be estimated. Its distribution, 
p(y), may be known on the basis of past experience. There is an- 
other possibility. The expected cost of error may itself be estimated, 
using an estimate of Y. This possibiiity is used in the biasing meihod 
presented beiow. The distribution, ply\Y), is provided for most esti- 
mators by the theory frona which it is derived, Finally, the cost-of- 
error function can be derived from the decision model in which the 
parameter appears, as was shown In the preceding section. 

When the parameters of a decision modeiiC.Fi, are known iritb cer- 
tabty, standard mathematical techniques can be applied to obtain a 
decision rule which will optimise tho perfonnance of the pven model. 
If the decision model has a umquc relaU\‘e minimum (or maximum) for 
fixed Yj, then Ihcro exist decision functiona Xt, = PtCTj) such that tho 
minimum value of /(A',-, F,) is equal to 

mYjh y,]- 

When tlic exact values of the parameters are not known (as is usu- 
ally the case), their values are estimated by some statistical proce- 
dure. Let tlie estimated values of Y/ be y;. Then, the actual per- 
formance of the model is given by 

flMi Yii. 

and the cost of error arising out of the use of tho estimated value.” is 
• Kolfl lt«l Uits cost is cquivslent to the “regret” ftssociaUd wUU caUmsUaS 
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The problem, then, is to reduce tlic cspcctcd cost of error, E (C) . This 
problem will be considered where the Y) are constant * but unknown, 
and the estittiates yj arc assumed to hare the probability distribution 
PiVilY,). 

The simplest decision situation involves ono control variable and 
one parameter; that is, 

v=nx.n 

In this case, cquatioa (7) becomes 

C’-Jb(v);Yi~MY)i n 

Lctp(i/iy) be the probability-density function of y witli mean Y and 
variance <r* Then, for any value of Y, tlie conditional expected cost 
of error is 

£<cin -fj/u(u)iyi-/bm;yiip(!im'iv, («) 

where the integration extends over the range r of the variable y. 

Afore generally, the conditional expected cost of error for a decision 
function d[y) of Uie observations is given by 

E(c\Y) Yi -mn nipcyin dy 

= i;|d(s(), Yl ( 9 ) 

where the functional L is defined by (9). The best estimating proce- 
dure, then, can be defined as a dccirion function d(y) which mini- 
mizes the total (unconditional) expected cost of error; 

E(c) - jr_f (sid(y), n ~ Mil, nipbi iipm dp dr m 

where S is the range of Y. 


A PROCEDURE FOR DERIVINC A BAYES ESTIMATE 

The purpose of using an optimal decision function d{y) can also 
be stated as that of aclueving cost reductions, 

£{( 70 /), Yi - Eid(3f), r{, ( 11 ) 

* Orel the period to wbicli a particubr dMwoa is to be appbed. 
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meaning: it is tho expected cost of error of taldng decision d when 
the observation is y, the expectation being taken over p(K). In 
other words, instead of a calculus of variations problem involvisg 
integration over all possible values of y, we need consider only the 
particular value observed. 

The posterior procedure, al^ugh less complex, requires derivation 
of an estimator for each different observed value. If the decision 
model which contains the parameter being estimated is to be used 
repetitively by a mathematically unsopliisticated decision maker, it 
may bo better to provide biro wilJi the simple results of the more com- 
plex prior procedure, than to have him solve the less complex integral 
equation each lime be observes a different value, y. In an example 
given in a later section (pp. 275-276), it will be seen that tiio prior 
procedure can yield results that are very easy to use. 


THE METHOD OF BIASF.D ESTIMATES • 

The biasing method differs from the Bayes estimating procedure is 
several respects. First, unlike the Bayes procedure, it assumes that 
the decUloD function d(y) has the same form as the decirion fusetian 
g(y} derived from the mode), assuming exact knowledge of all param* 
cter values. However, it modifies the function by adding to the estf* 
mate of K a biasing factor /J; therefore, d(y) takes the form g(\r -f 
Second, this method does not assume a priori knowledge of the 
dUtiibulion of Y and hence docs not seek to minimise the total ex- 
pected cost of error. However, it does reduce the cost of error for 
every possible true value of Y. This method, then, does not yield a 
best estimate, but it does yield a better estimate than is provided by 
the analogue or maximum-likelihood method. 

In the particular example we have been considering, equation (Di 
there is a decision function of the form 

d(y) = + ff), 

which is superior to the naive rule, y(y), in the sense of the criterion 
^ven in equation (11). For example, suppose that the estimates y 
have tho following probabiffty distribution, p(yjY), with mean Y = 3; 

y 1 2 3 4 5 

pO/in 0.15 0.15 0.35 0.25 O.IO 

■This procedure is due lo Shiv K. Gupta. 
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When the values of y from this distribution are used in the decision 
rule given iu equation (4), the expected cost of error turns out to be 
0.1114. 

Now suppose that wc keep the same decision rule but make a trans- 
formation y -t- 0 of the estimate y tmd use this transformed value to 
find the expected cost of error. Tliat is, when an estimate y is ob- 
tained, an adjusted value is used instead. The calculations 

shown in Table 8.8 tor severs} values of /} indicate that the exjjected 
cost of error can be reduced by tbU biasing procedure. 

This example siiowa the possibility of finding a decision function, 
d(j/) = g(.y -hflli which b auperior to tlm decision rule, g{y), for 
which the cost relation 

l{d{y).Y] (IS) 

holds (or all values of the parameter T. 

The method of biased estimates generahres (he example just con- 
sidered. An optimal dccieioo rule is sought in the fenn 

r.-d(v) = p(y-l-jS). (lOJ 

where o(y) is ^ dependent on the true value 

of 1'. It may, however, be dependent on an imblascd estimate of T, 
y. As a first step, a quantity fi is derived by carrying out the mini- 
mization 

minL|j?(y-l-fi), rj. (20) 

a 

However, the minimizing values of fi will in general be dependent on K: 

0 = /J(y). (21) 


Taolc S.S 

Value of ff Espectiid Cost of Error 


0 0.1114 

OJ 0.0790 

< 3 £ 0.0712 

0.6 0.0691 

0.S 0.06SS (mbtmuin) 

0.9 0.0703 

I.O Oi>727 
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is Eubsliluled toi y to obloin tio decision rule 

( 22 ) 


d(y) = »l» + dMl 


By deBiiition, it is certainly tnio that 

Ll|,Is + S(y)l.y| ^ '■Wi'). '’!• 


(23) 


However, it is not obvious that 

HJCv), I’l = 5^i- 


(2i) 


It should be noted that in (23) f remains 'r‘“‘’the°Tnlwa“n 
value y is constant. But in (21) fl vanes dome tile intesr 
p^Sss becaus. s is a variable. liquaUon (24) is proien i( 
stronger statement given by 

(25) 


it jl» + «(v)l, VI S 

1, proven. In a (cw speeiric eases (25) has been 
However, a general proof of equation (25) has not yet 
A set of sufficient conditions for equation (25) are: 

(a) The cost function 0(1', y) is convex, so that there is exactly 
one minimum. 

(b) The decision function fl(!/) » monotonic. 

(c) The biasing function fi(y) is moDotonic. 

(d) The function y + ‘3 increasing. 

(e) The condition 


di* 


Clffly + 0(y)], ri < 


( 26 ) 


is fulfilled for all 1’. , . , ohout 

(/) The probability-density function p(ylT) is sjTnmetr 
its mean T. 


These conditions presuppose that the function 0(y) has 
Ushed in explicit form. Under tins assumpUon, a” j^d,. 

easily verifiable with the posable exception of (e) ; . ,3 

tion (e) will normally be easier to verify in any specific cas 
equation (25). . , . 

The derivaUon of 0(y) according to (20) can be jcai 

lyHcally for only relatively ample models. In most cases 
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analysis is rcquirctl (wliich may be fncilitatcil by use of a computer). 

An approxiinatfi solution for /? may bo obtained by means of a 
Taylor approximation of the function 

Hy,Y) = CU{v),Y\. (27) 

Expanding about the point y = Y an«! neglecting terms beyond the 
third, we have 

ACy, Y) «. h{Y, Y) + (y~ YWiY, Y) 

(y - lO* (y - T)» 

+ h"(Y. Y) + h"'(Y. Y), (2S) 

where we have defined 

n - Mu, O. i - 2, 3. (23) 

d/ 

Of course, h'(y, E) » 0. 

IVe now r^pface y in (28) hy ff-i- fi. Alter /ntfidipiying (28) by 
UiQ probability distribution p(y|V) and integrating over y, Uic ex- 
pected cost associated with Iho biased decision rule 0(y + /S) becomes 

+ I'j - MY, Y) +J'S!±|riZv'(r, npCi/indj 

(30) 

Taking tlio partial derivative of (30) with respect to j3 and setting it 
equal to tero, we obtain 

<3* 

8h"(,Y, y) + “A"(y, Y)+^h"’{Y, y) = 0, (3i) 

where tr is the standard deviation of p(yjy)* This leads to Uio so- 
lution 



We have adopted the positive sign for the radical, since wc know timt 
/3 = 0 for cr =» 0. If h" :s> vh'". <t fitab-ordet Taylor expansion of the 


square root in (32) leads to 

d(y) « - 


^ h'"(Y,Y) - 

2 A"(y,y3 


( 33 ) 
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It should be noted that ^ in (33) is not dependent on any proper- 
ties of the probability distribution p(l/|i^) other than the standard 
deviation; furthermore, p is proportional to the skewness h"'/h" (at 
y = Y) oi the function h(jf, Y). By definition, h(y, Y) measures the 
cost associated with the naive decision rule. As one would expect, 
tlie optimal bias is zero when h{y, Y) is symmetrical about y ~ Y", 
that is, when the cost of overestimation is equal to the cost of under- 
estimation. 

The Taylor expazision (23) also proves to be a valuable tool in 
evaluating alternative decision rules. The term h{y, V) — h(K, y) 
represents the cost of error introduced by the naive decision rule. 
Equation (28) shows that U>c expected cost of error is approximated 
to the second degree by 

Clearly, the expected cost of error is an upper bound for the reduc- 
tion in expected cost which can be achieved by the optimal decision 
rule. Hence, the quantity S in (3-1) may be used as an optiadstjc 
estimate of savings. The actual savings will usuaUy be of the order 
of 5/2. 

Multiple Control Variables and Parameters 
We have considered only the simple decision model in which 

V = /(-Y, Y). 

but the more general form of a decision model is 

V^S{Xi,Y,\ i=l,2, 

y = 3, 2, • • •, mg. 

Kow we want to consider cases in which and are greater than !• 


Several Control Variables and One Parameter 

)Vhen V is a function of several control variables and one parameter, 
we have 

(35) 


l' = y(A'i.Xa 


17- 



Tlicn, for optimum V, we must set 
dV 
dXi' 
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for cacli X{. By solving these cquatious, wc obtain 
A'. = i= I, 2, ---.trti. 

If y b an estimated value of 1', then the estimated values of A'; are given 
by 

Substituting these values in equatiem (33), vc have 

(u): n- 

The cost of error C is, therefore, ©ven by 

C - J\sdv), M> • ■ ■, y) 

-m(y). ez(y>. ■ • yj - A(y. 19. 

It follows, lliercfore, that this particular geacraliration can be treated 
in the same manner as the case in which Uie model contains only one 
control variable and is a function of one parameter. That is, one 
biasing function, 0(y}, is all tliat is required. 

Several Parameters and Several Conbol Variables 

Since Uie case of several control variables can be reduced to one 
control variable, it is sufficient to consider the case of one control 
variable and more liian one parameter. In other words, the perform* 
aiicc model for this generalisation may be written in the form 

V = f(X; r». y = 1. 2, • • • , mi. (30) 

For optimum V, dV/dX «= 0; solving for X, we obtain 
A- = (r(r,. Vs. ■• ■,?„,). 

J/y/ijanesJimaied jvduocf J^ibcotieesliraatej value of A' is given by 
X = e(3/t, (37) 

*As9UJ]}iiig that the condition of aifficiwity u met. 
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and /(A", Y,) becomes 

/luCyi. !/2, ■ •,ym,)-,Yu I'l, ■■■, Y„,]. 

Tlio eo't of error (f) is, llieriTorc, given by 

C = f[Q{yi, 1/2, • ••,!/■,); Yu Y 2 , • • •» Y^,\ 

-SUYuY2.--‘.Y„;i-.Yu 1'2, y«.l 
“ KVi, I'j). y “ 1.2, 

If piViiYi) arc tlie probability-density functions of i//, then Die ex- 
pected cost of error is given by 

£(C) - jAfo; r,-)pMY,) daj. 

Now let y/ be biased by an amount &j, such Uiat E{C) given by 

E(o =]"«», 

is miDimum. 

We shall solve equation (38) approximately after makiog certain 
assumptions for the case io which i » 2. Then the gcncraliinbon 
from j a 2 to ; a in^ will become obvious. 

When ; s 2, equation (3$) becomes 

i'(c) - J Ato, + u„ V, + fti y„ K,)p(»ii ropcmi y,) <!»■ 

Now assume that; 

(1) All the probability-density (unctions p(y/lTi) are independent 
of each other. 

(2) The biasing factor fli will be found under the assumption tho 
all Yf’a (in our case Vj) except are known; that is, from cqua 
tion (37) 

X « y(yi, Y^. 

Ecjuation (39) then becomes 

B(C) =jMyi + )3,:V„K2)p(v,iy,)di/.. 

Now fii will be a function of both Yi and Yj. Since we have assumed 
Yt to be known, we shall substitute the value for Yz, [£(1/2) ® 
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and for Vj iU estimated value, so that j8i becomes a /unction of 
only. lienee, Similarly, we find Equa- 

tion (30) now becomes 

£(C) - f hi), + S,(„), „ + ftfe); 1',!',) 

pO/iI >’i)p(y2l i's) dyi di/2 (41) 

This procedure docs rctlucc llic expected cost of error, although it 
docs not yield the ojilitnum. 


An Exampfe of the Biasing hfethod 

Returning to the model given in equation (I), let p(i/,(r) be nor- 
mally distributed; tlint is, 

pO/ln - -;^-exp whenfy-rj<a 


where 


>0 


I 

V^2» fJr -9 


exp - 


othensdso 

(y-iy 


dl/. 


The ctet-of-error /unction (see (0)J after biasiog is given by 

Ml/ + y) - y(l/ + fi)-’* + (p + /»)» - 21'>‘ 

, - A(r -H d, y) + (y - m'(y -h g. y) +• • •+ /?», (43) 

w here Ji„ is Uic remainder after n terms and 




The expected cost of error is given by 

£(C)=J fi(y + /3,y)p(yll')dy 

* A(r + d.y) + /jA"(y + ft y) 

-f - ' • + -f- ft yj + ^sn. 

- ■ (,-w 




(y - n*- 


V^<r 


2^ 


-dy. 


(45) 
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\Vc solve equation (44) for 0 for various values of i’ such that £(C) 
is minimum. For a = 2o anti < OOOOfJl* Table 8.9 gives the 
values of 0 for several values of Y. 


Reduction of Variance of the Expected Cost of Error 

In the preceding discu'^ion 0, Utc biasing factor, was derived lo as 
to minimiw the expected co?l of error. We shall now find 0 so that 
Uic variance of the expected coal of error V(C) , given by 

v(C) = j" Ufa + A n - mi’pfai n t“> 

is rainimum. Equation (46) can be solved for 0, using various val- 
ues of y such tlial y(C) will be a tninimum; and, as was done be- 
fore, a relation can be found between 0 and y, that is. 


0 - 

For example, let the cosl-of-error function and the probability* 
density function be given by equations (42) and (43). TIjod equation 
(46) becomes 


/*>’+• 

(y-F)’, r h 

Jr-i ^ C5P -^S— ■'“J 


Tabu S.9 


6 0.78 

8 OiO 

la 0.46 

12 037 

14 0.33 



Table S.IO 
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Y fj 


6 ^ 

S .77 

10 .75 

12 .73 

14 .G3 


We solve equation (47) for p for various values of V sucli Oiat V(C) 
is ininimum. For a = 2a, Table S.IO gives the values of p for sev- 
eral values of 1*. 


A COMPAIUSON OF BAYES (PHIOR) 

AND BUSED ESTDUTORS 

The Bayes prior procedure was also applied to tJje simple cost mode) 
given is equation (1) 

K-.-Yy + i. 


wJdch may be jnterjiretcd as ao invcalory model leading to t)ie eco- 
nomic lot size: 


-V, 


Vr 




(48) 


Tim estimate y nas assumed to bnve a rectangular distribution with 
mean F and range 2iC « C; that is, 




forr-3<y < F + 3 
otlienvise. 


(49) 


These assumptions permit one to derive an analytical expre-nsion 
for UiG optimal biasing function: 

to - Vy'+A-’-ft yS3. (SO) 


Tiic details of derivation arc pven in Appendix IL Turning to Ibe 
Bayca method, it was assumed that the true value of the parameter 
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y is rcclangularly distributed inside tiie interval 3 2 ^= 1’ — 13. Tlie 
length of the interval is F = 10. This, then, was the integration in- 
terval for y in tlic Bayes fonnulation. As is also shown in Appendix 
II, the optimal-decision function Usadfy) may lio obtained in the 
following explicit form: 


i< 



forO^U^ 2A' 
(at2K<y^F 
for F < y < L, 


( 31 ) 
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peeled cost by biasiog and BaJ®* 
methods. 
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True values of V 

Figubs 83. Perccnfaso reOuctiou ot expected cost or error. 


whete i =s F 4- 2 /l. Tbo expected cost as a functioa of K was then 
calculated on a computer for the decision functions (ttO) aod (£1) as 
well as for the nais'e decision rule {45). Since the computer roulino 
is set up in terms of a Hosing function, the decision function (51) 
was first converted into its equivalent biasing function 0 (y) by the 


obvious relationship 





(M) 



273 SCIEXTinC METHOD 

The results ol the calculations are exhibited in Figures 8^, 8^, and 
8.4. 

The results show that in this particular example the Bayes method 
gives greater cost reductions (see Figure 8.2), especially in the lower 
range of F. It can also utilize additional information about Y, if it is 
available. For example, if the range of Y were further narrowed 
down, larger cost reducUons could be expected. The biasing method 
is quite indexible in this respecL Furthermore, the Bayes method 
usually requires less compulation. Its requirement for knowledge of 
the distribution of the true value of the parameter, however, may he 
difBcult to satisfy in practice. It should be obsers'cd that in this 
comparison the Bayes metliod assumed perfect knowledge of the dis- 
tribution of Y. If tlus distribution were known with error, as would 
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usually bo the case, the aclvantages of Uic Bayea method would be 
reduced. Therefore, as the error in ocir knowledge of p(T) increases, 
the relative advantage of the biasing method will also generally in- 
crease. 


CURVE FITTING 

Fitting a curve to a set of data is closely related to the estimation 
problem. Once the form of a curve (c-g., normal, exponential, or 
pol^momiol) has been selected, the remaining task is one of estimating 
its parameters, to which alt we have said about estimation is appli- 
cable. The problem of selecting Uie best form of a curve, however, is 
another matter. Most of what boa been written on curve fitting as- 
sumes that the researcher can make a good choice of the type of curve 
required either on the basis of a priori information or by examming 
a visual plot of the data. This assumption is probably justified 
for those who have a good vocabulary of functions at their commasd. 
Tbo task of picking the best form, however, lias hardly been discussed, 
let alone solved. There is no limit to the number of functions ahich 
eao be fitted to a set of data. TTiere is as yet no way of modeling 
this choice situation, and hence so way of making an optimal selec- 
tion. The best wc can do is to try different functional forms and 
compare the resulting dlstributians of estimating errors. The best 
form we can select by such a comparative procedure may not be 
good enough. Therefore, the researcher may ha\x to try a large 
number of forms before he finds one that is satisfactory. 

Fortunately, it has been shown, in many uses of distribution func- 
tions, tliat the resulting dblributions of errors arc not very sensitive 
to the form of (lie distribution function, within a fairly wide range 
of such types. 


CONCLUSION 

In this chapter wc Lava comidcrod the various ways of estimating 
the values of parameters which appear in a decision model and their 
relative advantages and disadvantages. Wo showed that the criteria 
usually applied to evaluating cstisaatiog procedures do not ncccsMriJy 
(and in fact seldom) minimite the total expected cost of error. Of the 
Uirce principal melhods of estimation — the analogue, maximum likcli- 
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mrc 3 mlh the addilional cxiicetcd cost d js leu 

tom UBe o! other proccdorea. The BayM po.tenor CBtoeW u 1 
complex that, the prior esttoaUrr, but ^odel 

Bliruator for carh d.Screut obrexacd value ” ™ ,iU„lv 

■uhich conlama the parameter bcios Ktimated “ ^ 

by a malicmatically unsopbiiticatol dccuion maUr, tl tM> 
to provide hiru uith tie sirupler resulta of the more compta pnor pro- 

' We also comrdcred a procedure of producins 
-althoush it docs riot mirdmirc the total ejpeeted i^t “ 
reduce it, corupared uilh uubiased cstrrtratcs for u 

of the parameter, ejeept m the rare case tchmi ‘•“= 
lytumelrieal. Cnlilce the Bayes procedures, it bcia} 

Sge of the distribuUou of the true values of lie P^' ^ 
estimated, but it docs use a value of the parameter based 
biased estimate of it. , . --cnor 

In one case, we saw that the Bayes pnor .^^asi* 

to the biased method, and that both arc supenor 
mum-likelihood estimate. No gencraliiaDens are justised b> 
case, but the results coohrm to what we would espeet over a 
problems. Again, it should be noted that the Bayes es^at 
comparison assumed errorless knowJ^ge of the 
parameter being estimated. If this dirtributioa is -j. 

the Bayes procedure might have done less wcU as compared 
biasing and marimum-Iikelihood methods. . 

An alternative to the cslimaling procedures we have de^n j 

sLsts of (1) adding the cost of estimating errors to the 

(2) using any estimating procedure for which p{ylV) _ 

(3) solving directly for the optimal value of X, the contro 

able. But even in the case of the toy model, this approach i» 
more complex mathematically than are the procedures we a^e 
sidered here. , ^ugrter 

For any of the estimating procodures we have consdered. 
one or more paramelcra is involved, the condirional P™ ^ 
p(yj]F/) play a criUcal role. It should be kept in mind a ^ 
probabilities are a function of Uie sample design and samp e 
Consequently, the estimation problem cannot be effectively ^ 

in isolation. Hie general problem is to select a sample design, 
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sample size, and an estimating procedure which minimize the sum of 
the iota! expected cost ol error and tlie cost of collecting the data and 
analyzing them. At present tliia problem can be ap[)roachcd only 
iteratively, by "oplimizuig” tiw parts of the i-csearch process sepa- 
rately, testing their combination, and trying out variations in (he 
design, sample size, and csUinaling procedure. The time rc<]uired 
for such comprehensive opliniizatioa is usually prohibitive. The 
dcsirabiiity of being able to find such optima in a practical way is so 
apparent that we can expect significant progress along these lines in 
tlie next few years. An effort in tbis direction involving a simplified 
type of estimation problem is discussed in the next chapter. 
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TESTING HYPOTtlESES 


INTRODUCTION 


Testing hypotheses is a special caso of estimation in which we 
are interested in determining which of two (and occasionally 
possible states exist. Where there arc only two courses of action (0i 
and Ci) open to the decision maker, he usually groups the possib e 
states into two classes (Ki and Vj) such that, if is tlie true statei 

Cl is the best course of action; otherwise Ca is. Quite frequently 
the states are described in terms of a single property whose 
either is or is not equal to some specified value. The research tas 
is to detennine whether Ti or Ta is the true state. These alter- 
natives are usually formulated aa hypotheses; for example, ffo- 
the true state is Ti, and Hx- the true state is not Ti. The fac 
that only two alteinalivcs are involved simplifies the estimation pro 
lem in some respects. This simplification permits us to take the 
question of sample size (and hence tlie cost of conducting the les 1 
explicitly Into account along with oUier questions associated with esti* 
mation. It is primarily to diow how the sample size and the cost o 
testing can be taken into consideration that this subject, testing 


hypotheses, is treated separately from estimation. 

Tests of hypotheses are useful in selectmg the best decision m 
problem situations that confront either a decision maker for whom 
the research is done or the reseand>er himself. On the one han , 
some practical problems involve only two possible comses of action- 
282 
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(■1) One variable changes or fails to change as another changes. 

(5) IVo or more qualitative properties {attributes) of elements 
in a populaUon are indcpcodcDi. 

(6) A specified distributioa "fits’* a set of data. 

(7) A set of observations is drawn at random from a single popu* 
lation. 

In the discussion that follows, two different conceptual systems 
are used in discussing testa of hypotlicscs. The more traditional one 
is due to NejTnan and Pearson (1933) and is generally used today 
as modified by \Yald (1942). This conceptual system is extensively 
employed in textbooks on statistics. It has considerable compnta* 
tional ad^'antages, but it is a difficult framework within which to 
make understandable Uic mcUiodological problems involved in by* 
pothesis testing. 

First, we shall take tlic approach of statistical-decision theory be* 
cause it exposes more effectively the nature of the underlying method- 
olopcal problems. However, it docs not generally yield practical pro- 
cedures for selecting a test. Then we will consider the more traditional 
Neyman^Pearson approach, in which we shall formulate a procedure 
of testing hypotheses which attempts to approximate optimality. 


THE APPilOACrt OF STATlSnCAL-DEClSIO.V 
THEORY TO HYPOTHESIS TESTING 

This approach may be formulated as follows. 

(1) Let Ai and Aj represent the courses of action which ore avail- 
able to the decision maker. Since io this approach we need not 
clasrify the possible states into two classes, let Fj, Ys, 
represent the possible states of nature. There may be an infinity of 
such states along a continuous scale, but we can consider a fimte 
number of cUscrete actions and states without loss of generahty 
However, the procedure to be described can be generalized to the 
infinite continuous case. Finally, let VfAilFj) represent the expect®*! 
value (or loss) to the decision maker of trying course of action Ai, 
where the true state is Yf. Now the test situation can be represented 
by a pay-off matrix such as is shown io Table 9 J. 

If the hypothesis-testing situation were to be treated as an uncer- 
tainty game against nature, U would be "solved” with only the infor- 
mation provided in a matrix such as is shown in Table 9.1. A mmi- 
max, generalized (Hurwicz) minimax, or minimax regret solution 
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TAPtK 0.!. PAT-Orr Matiux 

I’usaljk States o{ Nature 

Courbce — - — — - ■ - ■ 

ofActk'ii Yt 1* ••• >7 


Ai ri.i,ir,) rujro vw.ip^) 

Ai rusiP.) v'{A,ir,) vu,ii7) 


could he selected at Uiis point, liut if ob&cn'ations arc to be tnado 
os(J the rcduUing infonuutioa )3 to be used, a modified approach is re- 
quired. lu this ease no would proceed to the next step. 

(3) Let Vi, Vs, j/», ••• represent po&sib!o estimates o£ V’ (or 
some oUicr computation) obtained from a sample of obscrA'ations, 
and p(vtl^'i) represent the conditional probability of obtaining an 
csCiuuta of Pi nhcQ Y/ is the (me value of Y. Tlicsc conditional prob- 
abilities arc usually obtained from Utc Uicoo' ot sampling and cstl' 
motion employed and (in most eases) knowledge or an estimate of 
oDo or more parameters of the populaUon from ^vhich tlie sample is 
dr&\>’D. We will subsequently consider in detail how they are ob> 
Uincti It should be noted here, however, lliat the conditional prob- 
abilities dqicnd on the samplo siso and estimating procedure, both of 
wliicli must be selected by the researcher. 

The condiliosaJ probabilities can bo dispJayeil in a tabular form 
such ns is shown in Table 9.2. 

Tahle 9.3. CosomoMAU PnosAiuuTixa 


States of Nature 


nstimates 

Yi 

r, 

r, 



p(S/x\Yi) 


••• p(}fi\Yi) 


V> 

pMY>> 


vWi) 


S/» 



pCvdi’/) 


Sum 

Efvdih) - 1 

EpOoIK*) * 1 

rpfyilJ'*) - 1 
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Table 9.3. Possible SraATECiEa 
Possible Strategies 



Estimates Sj Si • " 


Ai .li ••• 

A\ Ai ••• At 


At At 'll 


Since the aet {v»} exluust all the possibilities, the sum of the pro - 
abilities in each column must equal 1. , 

(3) LctSj.Ss, represent the !!?«e 

able to the researcher. A strategy is a rule ^hi^ dictates the 
of action to be taken in tlic event of each possible estimate. i 
possible strategics are displayed in Table 9.3. . 

If, for example, only three estimates (yi, yi, and yt) P®” 
there would be eight possible alralcgles. These are shown in ® 


Clearly, if there are * possible estimates, there are 2' possible 
strale^es, which we shall designate as m. .of 

(4) Now we want to determine the probability that each course 
action will be selected in each strategy for each possible true va ue 
of Y,] that is, p(A<lS»,y,). These probabilities are J 

obtaining the courses of action associate with S» from Table 9-3 , ® 

the probabilities p(,Ai\Yi) are obtained from Table 9.2 and adde o 
each course of action. These computations yield a table of the >P 


Table 9.4. Fossisu: Stbateoies fob Thbeb Possible Estimates 



Si 

St 

s. 

St 

1 

s, 


St 

Vi 

Vi 

1 

Ai 

1 


At 

At 

At 

1 At ' 
At 

1 

Ai 

Ai 

1 

At 

Ai 

At 

At 1 
Ax 

At 

At 

At 



shovtQ in Table 9.3. For example, UMng Oie example in Table 0.4, 
tlio acUon probability associated wiUi St, /li, Tj U +■ 

p(l/a}F3) and, with Sj, ^1*, Vi, H is p(v»lFj). 

(5) Now for each combination (5*. Kj) tlie expected value, given 
Yi, is computed; that is. 

Y,) « Sp(.lil5*. VOlil Yii. (1) 

Tills yields values whicli can be diqilaycd as in Table 9.0. 


Txau: 9.6. Vaioes Associated wwu ItcsEARcnan’s Strategies 
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Table 9.6 is a revised pay-off matrix ia which expected gains or 
losses are shown. In Table 9.1 the pay-offs were simply gains or losses. 
1/ the probabilities of each Yf occurring are Icnown — that is^ p[Yi]~~ 
then the expected value of each strategy can be determined; 

EI'W - S p(r»ns», Y,). (2) 

i 

One could then select the strategy which maximizes expected value 
(or minimizes expected cost). If the p(V'i) are assumed to be un- 
known, wc again have an uncertainty game against nature, and such 
criteria as minimax, the generalized minimax, and minimax regret 
can be applied. 

It is also possible to select “mixed” strategics; that is, a selection 
of each of a set of simple strategics according to some specified prob- 
abilities. In any particular case the number of possible 
strategies is unlimited, since the number of mixes of probabilities is 
unlimited. In practice, however, it is usually possible, for reasons 
considered below, to reduce the number of strategies which should be 
considered. 

As Indicated, if on the basis of previously nvaibble (a priori) in- 
formation the probabilities of occurrence of possible states, ^ 
be. estimated, that selection of a strategy can be made which max- 
imises the expected I’alue. TTiat Is, for each Sk and Yf we can com- 
pute S Y,), which is the expected value of Sa, and select 

that 5 a for which this value ia tnaximum. If observations are made, the 
o priori probabilities can be adjusted to obtain o postmen probabilities 
on the basis of which the expected values are adjusted. These proce- 
dures, as one would expect from the discussion of estimation procedures, 
arc referred to as Ba’jee siraUgirt. 

Even if the probabilities pfK/) are not known, it is clear that I'C 
sliould, if possible, consider only strategies which maximize expected 
value for some set of possible values of pfTj). These may be called 
admissible strategies.* In some (relatively few) cases it is possible 
to identify all the adnussible strategics and hence restrict attention 
to them. 

Both the selection of admissible strategies and the calculation of 
o posteriori probabilities are illustrated in a very simple example of 

•As Wagner (1937) pomU out, “In malhemsucal statistica Uiere is s fine JiS" 
tmetion between the classes of admissblc strategies and Bayes strategies; fu^ 
ther, in special games no admissible strategiea may exist” (p. 8 fn). Tbis 
distinction and Umilalion are not oitical in discussion. 
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TaBIX 9.7. SfATlSTlCUS's Ia}SS£S 



States nf Nature 

i’osjble 1 

}'i lOdcfects 

r* = 20 dc/ecU 

.\ctiotis 

per IDO items 

jjcr 100 items 


1 

1 ® 

$20 

At 

$10 

0 


this approach given by Wa^cr (1057, pp. 13-17) n-Jiicfi is reproduced 
here with modification of only tiie sj-mbols employed. 

Small lots of a complex assembly item are to be subjected to an acceptance 
Bampling procedure. It is knona from experience that the number of defects 
per item occurs according to a PoisMo probability distnbution; and for the 
sake of zimphcity, it ia pi^tuiated here that Mature "produces" lots after 
seleotins & Poissoa distribution witli an average of cither 10 or SO defects 
per 100 items.* In the former ease, ibe lots are acceptable, and in the latter 
case unacceptable. CTeble 9 7J contains the stat^ician's pay-otf matrix. In 
this example, instead of represeotiog losses as negative numl^rs empioj-cJ in 
a naxinuring operation, they are treated as positive numUrs, and strategics 
Kbich minimize loss an to be invesUgiled. It is tunontil Uut thc^e mone- 
tary outcomea are good approdmations to the statistician's ["values’’]. 

The minimax etraCegy for (he stattsUclan, if he does no somplmg, u to se- 
lect At with probability H and Ag vritb probability li. Tlic expeciid value 
of the outcome, $6.(16, is then independent of Nature’s strategy. If the a 
priori probability p{Yi) » Vt and pfPs) « then Ai is the optimal [mini- 
ma.\ expected loss] action, giving an expected value of $3 00. 

Although the size of a sample is a vanable rrhich should be subject to eco- 
nomic analysis in a proposed statistical procedure, assumo that for various 
reasons only two items draum randomly out of the lot arc to bo inspected. 
The sample observations will be clasificd into three categories: i/j » 0 de- 
fect, ifj = I defect, Vi = 2 or more defects (if tno defects are found in ciflit r 
or teth items, inspection ceases). The eoaditionol proLobjlilics for y* are 
iJiown in Table 9.S.t 

There are 2 possible actions and 3 posiabla observations; hcncc 2’ >=3 
simple strategies exist, Table 0.9. Strafegy S^, for e.xample, EiwciCcd select- 
ing action Aj if Vi occun, and Aj othcntw^. 1/ Xi is the true stale of n.i- 

*\Vo utilize the disliaction empiored in <)uattty coolrot of defect vs. defective. 
Tfte 2tltef is iie&ssd iS tSfoif eJ ti* psrtuidsr nusabss zd alh’WAhle AfJivis fsr 
iteiss. 

tlf the Dumber of defects in iOO items has a PoUson dUtnbutioa with an 
average O, then it is postulated that the number of defects to two items is dis- 
tributed as a Poisson with an amoge 2g/t00. 
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Table 9.S. CoNDmoNAL Prodaeiuties 


Possible Observatioos ^ 

1 States of Nature 

y, 

Yt 

j/i = 0 defect in 2 items 

0.S2 

0.67 

yi = 1 defect in 2 items 

0.10 

0.27 

]/j = 2 or more detects in 2 itema 

0.02 

0.06 


Table 9.9. Simple StKATEaiES 



Table 9.10. Action Probadiutieb for Simple Stbateoie3 



luxe, then Vi occurs with probabibty Si, and consequently action A-i w 
wjOi probability .82. Table 9.10 pves the action probabilities ft"" 
strategy.* 

Finally Table 0.11 combines tbe previous matrices to give the expectea 


• The mathematician stoles that cath strategy defines a "mapping” {mm 
the sample space to the action qiace. 
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Table fi.ll. Average Ecokowc Evaluation in Dollabr 



ISO I &30 
UOI I &B0 


S.40 

S^Q 


a>eraeo lassca for each of the strategics. A first glance at Table 0.11 docs 
not reveal n'hkh strategies, if aDy, are inadmissible; a graphical analysis aids 
in the process, Figure 0.1 The expected losses are plotted os tvo coordinate 
points with reference to axes for F| and Kg- The bottom boundary, which 
is the lowest eonvex-ta*tlie>otigiQ boundary defined by strategy points, h.is 
the admissible strategics as vertices: <S|, $2. '$4i ^s * 

* An alternative delineation of admissible stralcBics niay be found in J. D. 
Wjlliami (1054, pp. 7J-7S>. 



FiccuO.!* Display orBtratc6>“- 
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The minimax strategy, found at the intersection of the bottom bounda^ 
and a 45® bne through the ori^, is to select S 4 with probabihty .46 and 
with probabihty .54. Given a priori probahilitks, the corresponding BayM 
strategy is found either by appi}nng the probabilities to Table 9.11 or by 
finding at which of the admissiWe strategies it is possible to construct toTigenf 
line with slope -p(I's)/p(i'i)- M p(I'i) =% ^nd p(I' 2 ) =M, Si is the 
optimal strategy. 

If a mimmax procedure la to he employed, it has been stated that the ex- 
pected loss without any data is $ 6 . 66 ; with data, the mioimax e.Tpected loss 
becomes $6.26. Therefore, it does not pay to take a sample of 2 items un- 
less the sampling ccKt is less than S.40.* In the case of piVi) = ^ 
pfl'a) without data the expected loss is $5.00, and with data is ^-<0. 
Eence with this a pnbn’ information it pays to inspect two items only if the 
cost of observation is less than S.30. Such conaderations are at the heart 
of seleclmg a single-stage samite eim or a sequential sampling procedure.t 
The use of a posteriori probabilities to arrive at a procedure identical to 
the admissible strategy defined in Table 9.11 is illustrotcd with pfl"}) ®** 
and pfVj) =» V*, for which £4 is optimal. 

If yi is observed (the a posteriori }vobal>ilityI, 

' rxW^+TxO^i ■ 

and p'(y0 ^ Table 9.7, A\ is found optimal. 

If Vs is obsened, 

~ j ~ xo.ifl^+' j°xo.g “ 

If Vi ia observed, 

- r xO.M'+ fxO.OC - «.«>. pVO - 0.50 

On Using a Bayes Strategy 

Before turning to the Neyman-Pearson framework it is important 
to observe that the use of Bayes stratc^es for testing problems is ex- 
tremely rare. Even enthusiasts for such strategics assert that they arc 
cumbersome to apply. In addition, they “involve the giving of ex- 
plicit loss functions, and still more restrictively, require the existence 
and knowledge of the o priori probability” [Chemoff and Moses 
(1959, p. 274)]. 

• This fitalement must be qualified if Uiere is some value in eoUeciing dflla. 
s.ny, for making a future estimate of Uie a pnori probabihlies 

t Aa the reader may verify, iocreasing the eample sue baa the eilect of lower- 
ing the boundary Ime in Figure 9a toward the ongm. But the marginal 
of successive obseia-aliona varies with the fwm of the probability distnbutioa, 
the sample size, and the q pricrri prdtwhihties. Hence, depending on the afore- 
mentioned considerations and data.piaces3iDg costs, it may, for example, pay 
to make two observations where it would not be economical to make one, 
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Although these strategies are cumbersome to apply, some simplifi- 
cation is possible, as wc shaU try to show by use of the Neyman- 
i’earson framework. In a later section we shall also show how the 
loss functions can he generated from a decision niodcl (following a 
procedure already discussed m Chapter 8). In connection with the 
difficulty that is said to he associated wiUi the a priori probability, it 
is useful to recall the discussion of Chapter 2 regarding uncertainty 
games. It was argued there tlat a pay-off matriv such as is shoma in 
Table 9.7 cannot be formulated wiUiout using or obtaining some 
knowlciige of the probabilities associated witli the various possible 
values of 1'; that is, pfK). This fact is hidden in many statistical- 
decision theoretic <liscussions of estimation and hypothesis testing 
because the data arc usually token as given. Few have inquired, as 
has Clmrrhman (1061), into wbat information is needed or assumed 
in the data-collection process. 

For example, consider Uie following scleclioo from the Wagner 
quotation gives In the last section: 

It it postuhlrJ here ttor Ksturc "proJMei" }t4s scleetlsg s Tmssoa 
distribution with an average of either 10 (F|) or 20 (fg) defects per 100 
items. 

How could one ever justify such an assuiuplion without having or 
assuming knowledge about p(F)7 \Y« are back in the black and white 
ball problem discussed in Chapter 2. That is, bow can we determise 
that a bowl contains only black and wbile balls without being able 
to learn something about the percentage of each? To be more specific, 
the hypothesis being tested makes an assertion about the value of F; 
for example, that it is I’l- The basia for this assertion is either Uieory 
or past observations, either of nhicli can be used to estimate the 
probvabilily-density function of F. 

Some knowledge or assumptions about the a priori probabilities 
are always present in formulating a testing problem. We will con- 
sider how' this knowledge can be used ia the discussloo that follows, 
in which wo will use the Ncyman-Pearson framework, but the problem 
of testing will be (ormulatcd in a way that is equivalent to that just 
presented. The discussion will, however, make more esqilicit the 
manner in which determination of sample size affects outcome. 


THE NEYMAN-PEABSON APPBOAai TO TESTING PROBLEMS 

The basic logic of tiie Jfcyinan-Pearson approach to the testing 
problem is revealed in the way the etatistical problem is formulated. 
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Using common sense, wo might ask, for example, "Is the true mean 
(/i) o! a population equal to aV' But in this approach we ask, “U 
the true mean of the population is equal to a, then how likely are we 
to obtain a given set of obscrvaUons?” If it U very likely that we 
would get the specified set of observations, we may accept the hy- 
pothesis, /i = a; otherwise we would reject it. Thus the task of 
statistical analysis requires Uelcrmiaing the probability of obtaining 
any given set of observations, assuming a specific hypothesis to be 
true; that is, of detcmilnlng the ability to make such 

determinations has been assumed in previous discussions; now we will 
see bow it is done. 

Suppose that wo arc testing the hypothesis that fi =* 100 and obtain 
an unbiased estimate of the mean equal to 101.2. Tlic task, then, is to 
detemune the probability of getting such an estimate if the true mean 
is equal to 100. To answer this we must know the probability-density 
function (PDF) of the estimated means. If we know or can make good 
estimates of the values of n and <r of the PDF of the element values, wo 
can determine the PDF of the estimated means. If the values of theele- 
meats are normally dbtribulcd, then the estimated means are also. 
(Where the PDF of tho element values is not normal, the PDF of tho 
estimated means tends toward normality as the sice of the sample from 
which the estimates are mads Increases.) The mean of the PDF of the 
estimated means is also >t. The variance of the estimated mean (v/)i 
however, is reduced. That U, if o' is the standard deviation of the popula- 
tion’s distribution, 



where x is computed from a ample random sample (of size n) with 
replacement, or, when drawn without replacement, 


J/ — no* 
W- In" 


(4) 


where N is the number of elements In the population. Thus the estimates 
become less variable as the sample rise increases, a fact we considered 
in Chapter 7. 

If « has a normal distribution, the probability that it exceeds any 
specified number k is given by 
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fu testing the ("null") hypotheas, = a, for a population whoso 
variance, is known, the usual procedure is as follows, pirst, a random 
sample of n items is dratvn from the population, and the sample mean, 
f, b computed. Let us say that £ = b. The deviation of b from a is con- 
verted into standard units, z*; 


b~a 



If b > o, then rt > 0; if 6 < o, then z* < 0; and if b = a, then zj = 0. 
Where /I =» a, the probability, p{i > h),is&vcn by equation (5). Tables 
of the normal curve give this probability if entered with the value of zj. 

One can also similarly determine the value of p(j < 6). if we let 
c = ( & — a I , wc can obtain the probability that x will deviate from a 
by more than c in either direction; it is 

p(f < a — c) + p(i > a + c) =5 J* /,(f, ji, ■>■*) di 

+ r (7) 

Ate we in a position yet to accept or reject the hypothesis, I/o'-u ~ at 
Not quite. How saiall should be the probability that £ deviates from a 
by the amount that it docs, assuming ^ » o, befom we reject the hy- 
pothesis? This is one of the methodological problems to wbicli we want 
to address ourselves. 

The minimum probability for whicli a hypothesis is accepted is 
called the significance level of the test and is symbolized by a. The 
value of a is conventionally token to be 0.05 or 0.01, but this choice 
generally has no rational basis. It is such a b.isis that wc seek lierc. 

First let us sec precisely what « means. If wc test ffv'M = a at a 
significance level a, tlicn we would reject f/o when it is true with a 
probability equal to a because of deviations of the estimate from a 
in a proportion of such csUinatcs. 

In testing the hypotliesis, Ho.> =a, with a 5{>cclSed <r, tlicre arc 
two ways we can proceed; with a ose-zided or a tieo-sided test, One- 
sided tests are shown in Figures 9.3(a) and 9.2(h), and Uic two-sided 
test is shown in Figure 9.2(c). 

For any sigtulicance level, a, there ia a point on the abscissa, Xa> which 
is Za standa^ units away from a, aucii that the area on one side of Xa 
is equal to a and on the other is equal to 1 — <». Let Xa"** represent 
Xa > o, and Xa“ represents x* < o. Then 
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>Area=sa 

(a) Oae-SidedTeet 



tjetBaJ 
(6) On«-Sid«dTMt 



(r| Two-Sid<dTe)( 


FiGcaE 02. One- nnJ IcsU 

of Ilf' >• =* a, naiuming *• = ° 


J* fn{£. P, vt)dx = a ^ ' 

an<i 

J* J-ix, Ji, vi) di = a. ^ ^ 

In UlolwoMdcd test weuscro.sa. 

That we arc inclined to reject a hypothesis when a rare c\-cn - - 
rare if the hypoUicsis is true) occurs is illustrated by the fo o 
coin-tossing example. Even if the probability of getting a 
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a loss ol a coin is wc Iiave Bome probability of getting 100 con* 
scciitivc hcaiis. But if \\c Jtd get this number of consecutive heads 
wo would be more likely to sus-pcct the coin tlian conclude that wc 
have observed a very rare event. One obvious tjuestion is, why not 
make a as sinull us possible so that there is praclically no probability 
of rejecting tlio byixithcsis when it is true? The answer lies in tlic 
fact Uiat anoUicr tj'jur of error can be made when testing a hypothesis 
and that this second Ijiic increases as the first decreases. 

Tlic error of rejecUiig a hypoUictia «hen it is true is called a type 
f error, and iU probability U cQual to «. A typo II error consists of 
accepting a hypotiicsis when it is false. As wc shall see, there is no 
simple measure of Uic probability of this type of error occurring be- 
cause it depends on what the true value of n is. If we are testing the 
hypothesis //«:/< = 100, uherc «r » 10, wc may be quite likely to ac- 
cept this hypothesis when /• is actually lOl, but very unlikely to do 
so when = 200. Furlitcnuorc, the probability of making any par- 
ticular tyjw If error decreases as the sample sue increases. 

Now suppose that wo arc conducting a two-sided test of the hy- 
pothesis //(Pm b a when >> is actually equal to b (bs^o). Tills situ- 
ation is represented in Figure 9-3. The unshaded area in this figure 
is equal to /f, the probability of accepting // 9 .> » a wlicn a b, and 
the shaded area is equal tot -0. To determine the value of 0 umlcr 
these conditions ue must first determine the values of and :/*. 



FioubeW. 
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These arc given by 




By Uic use of tables of the nonnal cun'e wc can now find 0, the pro^ 
ability of getting an observation i whicli falls between Zf and Sf » 
that is, 




Pul another way, sinco 


fjl'' •» +• o 


Xi* » + rt, 

then, when it ^h, ^ 

p(xfl“ < i < xg*) - r /.(f, M, O dJ- 

•'tg 

Hence 

p < - — - < " p{xg~ < i < xg*‘). 

Now let i (in i ■» W be a variable. Then w o can express the deviatioa 
of a from b in units of the population’s standard de%’iation, v. lAi ^ reP" 
resent this amount. That is, 

(17) 


Now, if we let K vary, by use of equation (15) we can obtain a func- 
tion which describes tbc type II error. This is called the 
junction of the testing proc^ure. The plot of this function yields 
what are called operating charaeteristic (OC) CT*rwes of the testing pJ'O" 
cedure. A set of such curves for the test we have described is shown m 
Figure 9.4. Since the curves are ^mmetrical, only one side is shown- 
From an examination of the OC curves as well as the power func- 
tion it is apparent that the probability of making any particular type 
II error decreases as the sample size and the significance level in- 
crease. 
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FiOt'RS 9.4. Op«raliae tbamteristic curves of tlis lwo*sulcd oonnal r<tcst. 


Th« Methodological Problenu Involved io Hj^thesU Testing 

Now we can restate in Uie Neyman>pearsoD framework the nature 
of tbfl mctliodological problem involved in tcslhig a hypothesis. The 
researcher must make threo decisions: Ute sample sue (») and the 
testing procedure to be used, and the eignificsoce level (a) at which 
Iho lest is to be perfoimcd. Ideally these detisioua should be made 
in such a way as to miniumc the sum ol tlie following costs: 

(1) C,; tie cost of making the obscia-ations. 

(2) Cai the co»t of Ibo analysis (performing the test). 

(3) E(Cs): the expected cost of typo I error. 

(4) E{Cii) : the expected cost of type II error. 

Expected Cost of Type I Error 

In order to detenniae the cost of a type I error we must know what 
action is taken if the hypo«M»w b«ng tested is reicctcd and what are 
the tonsequencca of Uiis rejection. For example, suppose that the 
hypothesis beins tested makes an assertion about Uie average value of 
a property of items made in Iota in a factory. A saoipJe of items 
from each lot is obsen-ed. If the hypothesis, //o:;* « o, is accepted, 
the lot is shipped to fl customer; otherwise it is dcstroj ocl. The cost of 
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* \ i« Uf r can usually be dclcnnincd without 
destroying an acceptable lot. C,. can y . ^ven by 

much difficulty. Then the expected cost of a t> pc I error i P 


EiCt) ^ nCi. 


(18) 


U, whoa a lot ia rojoctal, a hundred por coat ^woid 

coarplelo couat), Ihen-assumine no J^giho addi- 

cvcatually bo acccplcd il n - o. Hero C, is the cost or lasrug 
tional “ n) observations. Qunnosei 

The expected cost of a tjTio 1 error may bo °’“™ ““P’“' 
lor example, that 11 a lot U reiccted the aetron token “ “ 

value of the estimated mean, f, and that the cost of error 
off: that is, , (19) 


Then the expected cost of error tor a two-sided test would be 

£(Ct) = J” " d (*)/•(*. hr <*■* +X 


The corresponding values of £(£,) J’S'Shavc 

Consider the following variation of the sunple decision model n 
been using illustratively: 


= XP+-- 


(21) 


Suppose that we arc testing the hypothesis, //o:P “ a- 
estates of F. g, ^here g =. o + Aa. If tbehypothe^ ^ S 

pose that we substitute tor 7 the vidue of y on the of to 
rejection occurred. Then from prc^^ous analysis (p. 2U) we wiu 


Ct- 


= + Vg - 2V^ 


( 22 ) 


= -7= + Vg- 

Vg 

and, therefore, the expected cost of error for the two-sided test would be 

' P 


h Y. dy 


E(Ci) = J*™" - 2V?^fn{y, 1 

^ + Vg -2Vp)/n(g. P. dg. (23) 


Computation of the expected cost of a type I error is, ’ ^on 

complicated imder the following conditions. If each tune the 



USTLNO UyrOTHiSES 301 
rule is used, the hj-pothosU, *= a, fa retested, Uieii E{Ci) as given 
in (21) can be used. 1/, hou ever, after J? has been substituted for F.ne 
then test the Ijypothcsia, IlaiY & much more complcs silustiou 
arises. We shal] consider this rituation 5u Chaptci 13 in the discussion 
of model controls. 

Expected Cost of Type II Error 

Wlierc SVC test n hypolhcsfa coneeraing the value of a parameter, Y, 
in a decision mudcl, we c-an <lctcrmuic for each possible true value of 
F, a + 4a, the cost of accepting tho liypothcsis, //q : F =* a. Tho pro- 
cedure can Ijc illustrated by um; of the decidoti model 

r«Ar + ~ 

If P »> rt -f 4a and wo <isc this correct value for F, kc get 



Vo -h Aa 

(2i) 

and 


r, - 2v'«+^. 

(26) 

If we ubo P •» a ' 

,dien F ■ o -f Aa, we get 




(2C) 

and 



(27) 


Then the cost of a type II error is given by 

- V. (28) 

Vo 

In onler to determine Uje expected cost of typo H errors it is necessary 
to know tho probability-denaty function of 4a, p{4o). Since F = a -f 
Aa and o is a constant, this requirement fa equivalent to a knowledge of 
the probability-density function of F, p(^. If p(Aa) were kno'vn, the 
expected cost of typo If error would bo 


E{Cn) 


' pfAa) dAa. (29) 
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Ficuus Oi. Mioimal cosU vcmi« 






( 31 ) 

( 35 ) 


Tlien Uio value of ^ Is found for wlikh tlia equation holds. How difTicult 
this h dcpcfida on the I’DF of f). If iieccssirj-, approximatioas ran 
used. 


CONCLUSION 

The discussion of sampCng, cslimaUon, and hypothesis testinR cm* 
phasites the central role of mathematical statistics in modem rc- 
Fcarch. To many researchers, however, mathematical stnlislics is a 
mysterj’, an-ajysis of whidi they avmd by follotting "conr-entionar’ 
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procedures. As long as this is the case, inference itself is a mystery 
to them, no matter bow knowing they may appear to be. 

After one has struggled to an understanding of the methodological 
problems involved in afatistical tests of fiypothcsos and after oue 
realises how difficult even appro-ttmatcly* optimal solutions arc to ob' 
tain, he may sympathise with the practice of using conventional test- 
ing procedures without evaluating them. Yet long- (if not short-) 
run progress in science depends on our ability to improve these infer- 
ential procedures. We cannot evaluate conventional procedures unless 
we compare them in specific problems witli optimal or approximately 
optimal procedures. 

It is important to realise that the detailed analysis we have gone 
through here did not raise a single question that is not implicitly in- 
volved in conventional testing. It has been shown how a decision 
model can be used to obtain estimates of the costs of errors involved 
in testing, and how a decision model of the testing procedure itself 
may be constructed. Zo pure science one can seldom do other than 
assume relevant cost fusetiooj, but these assumptions should be made 
explicit ao that they are ojrca for future investigation. 

Computational complexity is perhaps the major obstacle to optimi- 
tatlon of testing procedures. Consequently future progress depends 
on developing better computational tccbniquee and faster computers 
particularly well suited for ecinplc.v integration. An alterantii'e that 
should be pursued simultaneously is the development of quick but 
efficient approximation techniques. 

Tests of hypotheses play a particularly important role in experi- 
mentation, which wc consider in tlic next chapter, and in testing and 
controlling the moilel, which wc consider in Chapter 13. 
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EXPERIMENTATION 
AND CORRELATION 


INTRODUCTION 

Jo the ]Ast five clinpters have confiJmd those aspects 0 / re* 
ecarch tvliich arc coneemed trjth cvalnaUng the parameters appeariog 
ia a model. It n-sa implicit!}' .assumed throughout this discussion tliat 
the parameters considerwl were relevant. Id this chapter we consider 
establishing or checking their relevance. 

Slodels eon fail to represent plieoomcoa adequatelj because they 
either contain irrelevant parameters or do not contain relcrant ones. 
A relevant parameter is a variable or constant which has a significant 
effect (probabilistically or deterministically) on the outcome vari.able. 

U'hcn the researcher rleals with u relatively simple phenomenon, his 
own analysis of it may reveal what variahles nntl constants atfect tiie 
outcome. For example, in analyzing the amount of savings a man 
has in the bank one can hlentify ns relevant (a) the amounts depos- 
ited, (6J the time of the deposits, (c) the amounts withdrawn, (d) 
the time of the withdrawals, an«l (e) the Interest rate and the way it is 
applied. If U)C relevance of the variables is not os obvious as it is 
in tills cx.ample, experimentation can be used to check it. 

On llic other hand, it the research involves a complex phenomenon 
wluch has liven extensively studied in Uw past, the researcher may 
“inherit” knowlcilgc of its structure. Nevertheless, he may want to 
check it by cxperimenlnlion. If, however, the atniclurc is unknown 
and too complex for analysis by inapeeUon, the researcher may have 
U1 
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to use experimentation to determine which of a set of variables arc 
relevant. Therefore, experimentation has two purposes: (a) to test 
the relationships asserted in a model once it is set up, and (b) to ex* 
ploro some possibilities before setting op a model. 

The discussion of models in Chapter 4 showed that we should seek 
decision models which contain controlled variables by the manipula- 
tion of which the decision maker can affect the outcome of his prob- 
lem. The model should also contain the uncontrolled variables which 
affect the outcome. It should assert the causal connections between 
these variables and the outcome. If analysis of the situation reveals 
these variables, as in the savings example, a model can be constructed 
without experimentation. If not, regression and correlation analysis 
can bo used to provide a sort of “candidate*’ variables, and experimen- 
tation can be used to detennine whether tlic)' affect the outcome, and 
how. 

It will be recalled from Chapter 1 that, in order to establish one 
variable as the cause or producer of another, we must determine 
whether one is necessary and/or sufficient for the otJ)er, TJic condi- 
tions for establishing necessity and sufficiency were discussed in a 
general way in Chapter 1. Now wc consider them in more detail. 

MILL’S CANONS OF rNDUCHON 

Tlic first systematic effort to specify the conditions under which, 
and operations by which, one tiling, A, can be said to be the cause of 
another, P, was made by J. B. Mill (1862) . He set down five "canons 
of induction," the first three of which were concerned with establish- 
ing one thing as tiic cause of another. We shall reformulate these 
rules in our own language. 


The Method of Agreement 

Let A, B, and <7 represent a eet of conditions or properties wliich 
describe a slate, and A', B', and C represent the absence of these 
conditions. Let P represent some specified phenomena. Then, ac- 
cording to Mil), if 

(а) P occurs under conditions A, B, and (7, and 

(б) P occurs under conditions A, O', and C', 

then A is the cause of P. On the basis of these tests, however, one 
could not say that A is Eufficicni for P. 
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First, it is necessary to knqw that A, B, and C completely describe 
the state. Otherwise there may be another condition, D, which U 
present in both tests and which causes P, rather than A. Thus we 
require an exhaustive listing of every possible cause of P. In prac- 
tice it is hard to conceive of a situation in which such a list can be 
provided. 

Second, assume that d, H, and C cxliaust all possible state condi- 
tions, The two teats (a) and (6) are not sulGcicnt to establish suffi- 
ciency even under the«e condiliona. Consider the following example. 
Let A, li, and C represent milk, bread, and butter, and P be an al- 
lergic reaction, s.ay hives. Then, if hives follow a meal of the Uiree 
and also follow a meal of only milk, in the environment involved, 
milk would appear to be sufficient for hives, if hives always followed 
milk. Yet we would not w.ant to call milk a cause of hives unless we 
knew that hives would not occur unless milk were taken. If hives 
occurred even if no food were eaten, then clearly milk would not be 
Uie cause of hives; or if hives followed any food but did not occur 
if nothing were eaten, we would call food, not milk, the cause of the 
hive*. 

To nseert tlial A is sufficient for P can he interpreted to mean “A, 
by lUclf Is enough to assure P.” Hut generally we mean eomcUiiag 
stronger: “A will he followed by P under any set of conditions.’' 
Mill's test does not establish sufficiency in this sense even if A, B, 
And C exhaust the conditions. To cstablisli sufficiency in this sense 
we would first liavc to run the following tests with Uio indicated 
resulbi: 

A, B, C, followed by P 
.rl.P'.C followed hyp 
A, B, C followed by P 
A, B', C followed by P. 

That is, we would have to oliwrve A in evco' combination of B and C 
and their absence. We wouW also have to observ’C at least one in- 
stance in which A’ is not followe*! by P- We must do this in order 
to establish tiiat P <loes not occur under any conditions. If U did, 
the concept of sufficiency would have no BigniScance relative to it. 

Finally, the tests assume that conditions A, B, and C are two- 
valued la the sense that they arc eillicr present or absent. This is 
not always the case. Suppo«e, for example, that we want to deter- 
mine whether temperature affccta the length of a metal bar. We 
cannot remove temperature; we can onty change it. That is, even 
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the tests we have specified are not adequate where quantitative vari- 
ables are involved. 

The Method of Difference 

If A, S, and C are foilowed bj' P, and A', B, and C arc not fol- 
lowed by P, then, according to Mill, A can be said to cause P. 

Here the test is designed to determine whether A is nece$sary for 
the occurrence of P. The test does not accomplish its purpose, for 
essentially the same reasons Uiat the first method fails. Again we 
would have to know that A, B, and C exhaust all possible conditions, 
since the occurrence and nonoccurrence of P may be due to the 
presence of D in the first instance and its absence in the second. The 
test also assumes that A can be removed completely. Finally, in 
order to determine whether A is completely necessary even if B and C 
completely specify the state, we would have to show that under any 
conditions in which A does not occur, P does not occur. That is 

A, £?,C followed by P 
A\ B,C not followed by P 
A', B', C not followed by P 
A', B, C not followed by P 
A', S', C' not followed by P. 


The Joint Method 

This combines Uje first two methods into the following tests; 

A, 5. C followed by P 
A, B’, C' frflowed by P 
A', B, C not followed by P. 

This is supposed to establish A as a necessary and sufficient condition 
for P. It fails to do so for the reasons already cited. If we now 
combine the two sets that we (not Jtill) said were required, we 
would get 

A, B, C followed by P 
A, B',C followed by P 
A, B, C fdlonnd by P 
A, B', C followed by P 
A', B, C not followed by P 
A', B', C not fdiowed by P 
A', B, C not followed by P 
A', B', C not followed by P. 
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In general, it can be ghown that, if n conditions are involved, tlieu 2" 
testa would be necessary to establish this strong relationship. 

Wlien these tests arc displayed as we have done, an important ob- 
servation can be made. If we want to determine whether B or C is a 
nccess.ary an<i/or gufficienl condition for I*, no additional tests are re- 
quired. since we have D and B’ in alt possible combinations of A and 
C, and C and C in all possible combinations of A and B. That is, if 
such tcfcting did establish that A ia or is not n cause of P, it would do 
the same for U an<i C. Tliia observation, that we can answer many 
cnus.al questions simultaneously, played a key role in the dcvclop- 
raont of modern experimental design, which wc wilt di«cuS8 below. 

CoDcctiog our other comments, we can also obser\-e tJist an ade- 
quate testing procedure ehouM 

(1) acknowledge the jneomplctencas of any listing of ronditioiis 
and account for the undesign.alci! (and bcncc uncontrolled) condi- 
tions, and 

(2) allow for quantitative variables as well as qualitative ones. 

Satisfaction of the first of these conditions, os wc shall sec, makes 

for statistically designed cxpcrifflcnta rather (ban (he deterministic 
ones of Mill. Tlic imrertance of Mill’s contribution, however, should 
not be minimized because wc “know belter" tod.sy. He made a very 
important step toward tlie development of a mcthodologj* of experi- 
mental design. The contemporary concept of cspcriinentaiion, to 
which wc cow turn, is largely due to It. A. Fisher (IMS and 1919). 


MODEHN EXPERfMENTATION'— TIIE ANALYSIS OF t'AniANCE 

In Mill’s canons each condition or variable wa.s considered to be 
two valued: present or absent. But now let us consider two con- 
trolled variables A and B, and an outcome variable T, which may 
have more than two values. Tliey may bo temperature and atmos- 
pheric pressure or height and age. Suppose that the scales on which 
these variables are measured arc divided into three intervals: uj, oj, 
flj, and bti bsi f'a- Then we can show all possible combinations of 
these variable intervals in tabular forov Table 10.1 slum-s nine pos- 
siblo situations. In gencr.al, if A’$ scale were divided into m intervals 
and B’s into n intervals, there would lie »t« jwssihlc situations. 

Let Aj be the average of the r-valucs in row Oj, and similarly for 
As, and As. Correspondingly, let Bi, Bt, and Bs be the average of 
tbo F-values in each column. Kach rolnmn average represents a set 



31G SCIEMTiriC MirniOD 


Tabix 10.1 


bl bf tj 


"l 


of ob«erYnlions taken un»Icr llic Mmc conditions; llicn (under some 
ftf^umptions which arc tlisru'-sctl Wlow) the differences between col- 
umn nveraRCs must be due to the difference in column valuc.s. I^t 
<re’ represent the variance of the column averages and a,* the variance 
of the row averages. Let ct' represent the total variance between 
the nine obscia’alions. If the column variable and tlie row vari.vbte 
were Uic only two variables which affect V, then the total variance 
would limply be equal to the sum of the vari.anccs due to these two 
variables; that Is, 

.r’ (I) 

But if there arc other variables contributing to the aTiriancc, then 

cr’ > + «#*• (2) 

Then sr* + «<*) Is the variance due to urjcontrolled variables. 
Let CK* represent this “residua!” or "cbance” variance. If the vari- 
ance due to Uie row or column variables is no larger than that due to 
the uncontrolled variables (under assumptions to be listed below), 
(hen the S'ariations in T’ accompanying changes in A and B arc no 
larger than one would expect by “chance." Hence, the effect of the 
controlled variables is not significant If these variances arc larger 
tlian the residual variance, then the effect of these variahlcs is sig- 
niffcant 

The variances to wluch we have been referring arc never known 
but are estimated on the basis of a sample of possible observations in 
each cell. On the basis of these estimates we can test the two hy- 
potheses: 

V * > »«* 

<r/ > 

This anafysis of variance is based on the distribution of estimated 
variances about true variances of normal populations: the F-dis* 
tributioD. 
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The three assumptions of this proc«iurc can be stated as lollous; 

(1) Any observation can be considered to be the sum of four com- 
ponents: (o) a constant component which is not affected by the col- 
umn or row variables; (b) a component due to the possible influence 
of the column variable; (c) a component due to the possible in- 
fluence of the row variable; and (d) a component due to the residxial 
variables. 

(2) The observations are alt drawn from normal populations with 
the same variances. 

(3) The residual effects are independent and normaliy distributed 
with scro mean (i.e., in the long run tlie negative and positive effects 
cancel out) and with the same variance for all observations. 

The effects of deviations from these assumptions have been studied 
extensively by statisticians with the general result of showing that 
they ore not as restrictive as they might appear. For discussion of 
this problem see Cochran (1(^17) and Eisenhart (1M7). 

Acceptance of the hypotheses, »,* > does not by itself establish 
the column variable as a producer of changes in the phenomenon, V. 
At best, lb does so only if the changes in values of tl)e column van* 
able precede those in V; similarly, for the row variable. 

Replication and Interaction 

Xow suppose that in cacli of the nine cells in the illustrative experi- 
mental design two or more observations arc taken; Uiat is, rcpfico- 
iions are made. Then another variance is involved; the vtithuucell 
variance. This variance is due to variations in the uncontrolled 
(residual) variables. Let this witlim-cell variance be v.*. \Vc may 
find now tliat 

or’ > »•* + »r’ + ffw®. (3) 

The remaining vari.ance {at* — v,' — wr* — must be due to some- 
thing we have not yet considered. It is duo to the interaction of the 
row and column variables. Two variables interact If the effect wliieli 
one has on the dependent phcoouena depends on the value of Uie 
other. For example, temperature ond atmospheric pressure interact 
relative to the boiling point of water. 

It is clear, then, that the residual variance, estiraale<l where only 
one obscrs'ation is made per cell, may contain variance due to inter- 
actions. But these can be separated only if the variance due to un- 
controlled variables can be computed directly from witbln-ccH vari- 
ance rather than determined as a ‘'remainder." 
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It should be noted that the classical experimental designs of Mill 
took no account of interactions. Further, while modern experimental 
procedures take the uncontrolled variables into account, they do not 
assume an exhaustive listing of all possible producers of a product. 
They also take quantitative variables and the statistical character 
of obsen’ations into account. 

In order to explore the concept of interaction more completely con- 
sider a simple wperiment involving two variables — or factors, as they 
are normally called — each at two values or levels. The factors arc A 
and B with levels ni and a*, and bj and bi, respectively. Each of 
the four possible combinations of ri-valucs and U-values is called a 
treatment These are aibj, Cabi» aibj, and ojbj. See Table 10.2. As- 
sume that an equal number of observations, n (n^l), is made in 


Table 102. Scneiarw DfsetAV of Simfle aa’d Hais Effects 
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each coll; tJjat is, for each trealmenL Lei the means ol the ohsen’a- 
tions be represented by 1, a, b, and ah; that is, o and b represent 
tfie second levels, the absence ol them represents the first level, and 1 
represents the absence of l>oth. 

Tije simph effects of a factor arc its elTccts at each level of the 
other factors. lienee, they arc the dijfercnfc* of the averages of llie 
observ-ations in tlic cells differentiated only by levels of the factor 
whose simple effect is being determined. Tliesc arc as follmvs: 


Simple effect of ./I (B at first level) « a — 1. (4) 

Simple effect of H (B at second level) = ol — h. (5) 

Simple effect of B (A at first level) * b — 1. (0) 

Simple effect of B (A at second level) » ob — a (7) 

The main of a factor is the average of Us simple effects; 

Main effect of A ■ §(a — I + o6 — l>). (8) 

Main effect of B ™ ^{b — 1 + ab — o). (0) 


If tile effects of the faeton are independent, then tl^o main effect of 
the factors is the best estimate of tlicir effects because their standard 
errors are 1/V^ times the standard error of the simple effects. It the 
factors arc independent, it also follows that the observed differences 
between their simple effects are due to random (uneontrolJed) fluctua- 
tions. Consequently to test for independence (and hence interaction) 
of the factors we test the hypothesis that the difference between sim- 
ple effects is equal to zero (using the MestJ. That is, if 

(flb - b) - (a - 1) « 0, (10) 

then A is indcpwident of B. Since the difference between the simple 
effects of D Is equal to that of A, 

(«b-o) - (b- J) « (ftb-W - (o- 1), (11) 

this also implies that B is independent of A. If these factors are not 
independent, then the difference fob — o — h4-l)j usually divided 
by a numerical factor, is a roc.asure of U«is ;frst-ordcr interaction, AB. 

Now consider an experiment in which three factors (A, B, and C) 
are involved, each at two levels. This « represented in Table lOA(a). 
Tables J0.3(b) and 10.3(c) show the results for A-D combinations at 
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Table 10 3. A Three-Factor Two-Level Experuien-t 
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(e) C at second level 

th«! C| and C- levels, respectively. It can be seen that the inter- 
action AB (C at first level) is a function of 

(fll. - l») - (o - 1) = Ob - 6 - o + 1 (12) 

and, the interaction AB (C at eecrnid level) b a function of 

{ahe — fcc) — (oc — «) = abe — be — ae •¥ c. (13) 

nie interaction AB is a function of the a%^rage of these; 

^(oic — 6c — oc + c oh — 6 — o 1). (N) 
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By similar algebra it can ix shown tliat tlip interaction AC is a func- 
tion of 

|(of>c — bt-i-m — e — ab-i-h— a + 1) (15) 

and the interaction DC ia a function of 

|(aic + 6c — ac — e — fit — li-fo + l). (16) 

The interaction of AB mth C is a function of 
dfl (C at second level) — AB {C at first level) 

= ^{abc — tc — ocd-c — flh-fh-ha — 1). (17) 

The interactions of AC with B and BC with A arc functiona of the 
same expression. Tliia, then, is called the uctynA-orier inlcraction, 
ABC. By extension, higber«order interactions may be obtained when 
more than three factors are involved. 


EXPEniSfENTAL DESIGN 

Atodern cxpcrlmenlai techniques allow us to design e.xperfinettts 
tliat elTectiveJy serve our objectives. Eac)) oS the large variety of ex- 
perimental designs which arc available was created to suit a particu- 
lar type of experimental situation. We will consider some of the more 
important of these designs and their relative advantagw and di.s- 
ndranlagcs. 


Factorial Designs 

All the designs which have been iise<i illustratively up to this point 
(Tables 10.1, 10.2, and 10.3) are /actorinf designs. In a factorLal de- 
sign observations arc made for each possible combination of factor 
levels. A factorial design involving two factors each at two levefi 
(Table lOB) is called a 2* factorial design since It has 4 eclls. Three 
factors at two levels each (Table 10.3) yield a 2* factorial design with 
8 cells. The factors need not have eqtial numbers of levels. If A 
has 2 levels, B has 3 levels, and C has 4 levels, there nill be 2 X 3 X 4 
= 24 cells. 

Cochran and Cox (1957, pp. 150-151) summarize the advantages 
of factorial designs as follows: 
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The advantages of factorial expenmentatlon naturally depend on the pur- 
pose of the experiment. "We suppose for the present that the purpose is to 
investigate the effects of each factor over some preassigned range that is 
covered by the levels of that factor wHcli are used in the experiment. In 
other words the object is to obtain a broa«l picture of the effects of the fac- 
tors rather than to find, say, the combination of the levels of the factors 
that give a maximum response. One procc*lorc js to conduct separate ex- 
periments each of which deals only with a single factor. Another is to in- 
clude all factors simultancoudy by means of a factorial experiment. 

If all factors are independent in thdr effects, the factorial approach will 
result in a considerable saving of the time and material devoted to the ex- 
periments. The saving follows from two facts. First . . . when factors 
are independent all the simple effects of a factor are equal to its main effect, 
so that the main effects are the only quantities needed to describe fully the 
consequences of variations in the factor. Secondly, in a factorial experiment, 
each main effect is estimated with the same precision as li the whole experi- 
ment had been devoted to that factor alone. ... If there were n factors, 
all at two levels and all independent, the single factor approach would ne- 
cessitate n times as much experimental matenal as a factorial arrangement 
of equal precision. 

Practical eonsidcratioos may diminish tbU gain. The experimenter fre- 
quently lacks the resources to conduct a large experiment wjth many treal- 
mentsj and must proceed with only one or two factors at a time. Further 
... as the number of treatment combinations b an experiment ia enlarged, 
the atandard error per unit increases. This standard error is therefore likely 
to bo higher for a large factorial e.xperiment than for a comparable sbgle- 
factor experiment. Thia berease b stan<Lird error can usually be kept mall 
by the device known as coafoundbg [discussed below]. . . . 

The value of factorial experiments when the factors are not inde- 
pendent is Bummariied by Cochran and Cox {1957, p. 361), as follows: 

IVlieB factors are not bdependcot, the simple effects of a factor vary ac- 
cording to the particular combination of the other factors with which these 
are produced. In thia ease the smgle-factor approach is likely to provide 
only a number of disconnected pieces of iaformatton that cannot easily be 
put together. In order to conduct an experiment on a single factor A, some 
decision must be made about the Icveb of other factors B, C, D, say, tint are 
io be used in the experiment. . . . Tlie expenment reveals the effects of A 
for this particular combination of B, C, and D, but no information is pro- 
vided for predicting the effects of A with any other combination of D, C, and 
D. With a factorial approach, on the other liand, the effects of A are ex- 
amined for every combination of B, C, and D that is included in the e-xpen- 
ment. Thus a great deal of mfonnation is accumulated both about the 
effects of the factors and about their mlerrehtiosships. 


Latin-Squaie Designs 

If three or more factors are considered to be independent and hence 
interactions are of no concern, then the factorial design may be modi- 
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Table 10.4. A LAti.'f'SQiiAiu; Desicn 
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fi 
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Cx 
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fitd so fls to require fewer obserx’atioiui in order to oiitain estimates 
of main effects, but ot a loss of precision in obtaining tliesc estimates. 

Suppose, for exampfe, lliat we luave tliree factors, A, B, and C, eacli 
at three levels. In place of a 3* (27-rell) design ire cna use a 9-ce)l 
design such ns is shown in Table 10.4. This is called a Latin square. 
In Table 10.4 each cell is defined by 3 values. Each value of C 
appears exactly once in each column and row. Hence, each value of 
A is obsen'ed under each vatiie of B and C, and (he same holds for 
Ji and C. That it holds for C is moro easily seen by recasting Table 
104 into its equivalent, which U shovn in Table 105. Note that al- 
though ( 0 |, bt, cj), for example, is not included, (ri, b\), (oi, Cg), 
and (6|, cs) arc. 

Now averages of tlie observations obt.sincd for each value of all 
three variables can be dclcrmincd, the corresponding variances esti- 
mated, and so on. In this type ot design (unlike factorial designs) 
each factor must have the same number of levels. 

Crcco-Latin squares 

IVe can even go one step further by introducing a fourth v-ariable 
D with 3 values, d|, dj, and dj. With 9 cells we can still s.itisfy the 
requirements of oljserving each level of eaeli variable for all values of 
the others. Such a design, the Greco-Latia square, is shown in Table 
10.6. In Ibis design each value of D also appears exactly once in 


Table 10.6. A Iiatik-S(iuaub Design 
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Table 10.0. A Gaeco-I^TtSf SafABE 
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each column and row and never ai»pc.ars twice with the inmc value of 
C. 

The further Ba\nng8 of ohscr\'at>on9 yielded by the Grcco-Latin 
square over the Latin square arc paid for by a further loss of precision 
in the estimates of the main effects of the factors. 

The three designs just considered arc all directed toward analysis 
of factor-produced effects. In many eases we arc interested in the 
effects of treatments wliich consist of eorobinations of factor levels 
and not in the effects of the factors taken separately, or the cffccU of 
the factors on each other. Tlic next two designs are particularly well 
suited for study of the effects of eomhinntloDS of factors. 


Completely Randomized Designs 

In this type of design the treatments arc assigned to tI)C units 
to be obscr%'ed at random. For example, suppose that there arc four 
treatments (Ci, ts, ts, and t«), and four replications are desired. Then 
sixteen units are required. Tliese units are numbered in a random 
order by use of a table of random numbers; for example, 

14,11,16,10 7,6,15,2 13,3,8,4 12,9,1,6 

' h ' h ' Ti ' ' U " 

The treatments are applied to the units as indicated and in the order 
shown. 

The advantages of this t3rpe of design arc: 

(1) Any number of treatments can be used, and any number of 
replications, not necessarily eqnal for each treatment, can also be 
used. This makes it posdble to utilize all the experiments which 
can be conducted, even where the number possible is not a multiple of 
the number of treatments. This is a particularly valuable property 
in small exploratory experiments. 
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(2) The statistical analysis of results is easy even if some observa« 
lions arc lost (rejected or miaing). The relative loss of information 
line to lost observations is rsiniraized. 

The principal disadvantages of tJiis type of design are: 

(f) Since tlie units receiving the same treatment may be different, 
a large experimental error may be ioeurred, but flijs ean be estimated 
accurately. For this reason, the effectiveness of this design increases 
with increased homogeneity of the objects or events observed. 

(2) Although this type of design is well euited tor the laboratory, 
it may be cumbersome and inefficient to use in the field, wliero ran- 
domized block designs arc generally preferable. 

Randomized Block Designs 

In this design the units to be obscr\-e<l arc grouped (as homo- 
geneously as fiossible) into "Woeks.” Treatments are then assigned 
to units in each block at random. For example, suppose that a 
sample of 5 items is taken from Uic output of 3 identical machines, 
I^et the units in each of these groups of S Items be numbered from I 
to 5, These S treatmenU may be assigned by use of random numbers 
as shown in Table 10.7. If there Is a difference between tbe three 
groups, and hence the machines, this can also be determined from 
tills design; that is, the replieations eao be eoasidered as values of a 
second experimental variable, the machine, and hence, if so considcrctl, 
this design is equivalent to a factorial design with no replications. 

The principal advantage of this type of design is that it generally 
yields less experimental error than does the completely randomized 
design. There is no limit to the number of treatraenU nnd replications 
which can be included. 


Taulk 107. A IlANOOMizen Block Drsins 
Replicate 

Trcntment 12 3 


1 5 S 2 

2 4 13 

3 13 4 

4 3 4 1 

5 2 2 3 
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Cochran and Cox (1957, pp. 106-107) auraraarize the advantages of 
this type of design as follows; 

1. By means of the grouping, more accurate results arc usually obtained 
than with completely randomized designs. 

2. Any number of treatments and any number of replicates may be in- 
cluded. . . . 

3. The statistical analj’Eis is str^htforward. Mishaps which necessitate 
the omission of a complete group or of the entire data from one or more 
treatments do not introduce any Mmiplication in the analj’sis. When data 
from some individual units are bcldng, the "missing-plot” technique devel- 
oped by Yates enables the available results to be fully utilized. Some ertra 
computational labor js, however, invoh'ed, aiul if the gaps arc numerous the 
design is less convenient in this respect than complete randomization. 

4. If the experimental error variance is larger for some treatments than 
for others, an unbiased error for testing any specific combmation of the 
treatment means can still be obtained. 


Confounding 

Consider a 2’ experiment Involving llirec factors, 4, B, and C, each 
at two levels. If this w-ere to be conducted in a randomized block de- 
sign with three replicates for each treatment, it would appear as shown 
in Table 10.8. If we are wilting to sacrifice accuracy in estimating 
higher-order interactions, the size of the blocks can be reduced. 

The highest-order interaction, ABC, was shown in equation (17) 
to be a function of 

abc -t-a + b + c — «b — ac-~6c — 1. 

Each replicate can now be divided into two blocks with four treat- 
ments each, as is shown in Table 10.9. The difference between the 


Tabi£ 10.8. A 2’ llAXDOiuzED BuJCa Design 
T reatments 12 3 
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Taole iO.9. A 2’ BxFEmuENT WITH ABC Completely Confocsbeo 
Hcplicatfi I n III 

Block 1 2 3 4 5 6 



tots! of Blocks 1, 3, and 5 sod the total of 2, 4, and 6 provides tlie 
basis for estimating the ABC inlcraction, and hence this interaction 
U eaid to be completeli/ confounded with the blocks. On the other 
hand, it can be shown that the effccta of A, D, C, AB, AC, and BC 
are independent of block differences and hence arc nneonfounded with 
blocks and depend only on tcithin-biock comparifons. The error in 
ostusating the ABC effect will generally be larger than in estimating 
any of the other effects. 

It will be recalled tiial the first-order Interactions were functions 
of (he following expressions: 

ABi flbe + c + ol»+I““l>c — oc — b — a 
ACi abc d-ac + b + I— bc — c — ob — o 
SCs abc 4-bc+o + l — cc — e—flb — b. 

An experiment can be designed with each of these interactions con- 
founded in one replieatioo. Such a design, with partial confounding, 
is shown in Tnble 10.10. Tlie interaction AB is a function of the dif- 
ference between blocks 1 and 2, AC of the differtnee between blocks 
3 and 4, and EC of the difference between blocks 5 and 6. The effects 
of A, B, C, and ABC depend on within-Wock differences. 


Table lO.tO. A 2* Expebiment wrrn AB, AC, anb BC 
PAnmtiT CoarotwnED 
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[The fldvaniage of confounding] comes from tlie reduction in the cxpcri- 
menial error by the u»e of a Uoek which is more homogeneous, or whieU 
can be subjected to & more uniform technique, than the complete replicate. 
. . . The disadvantages of confounding consist of (1) the reduction in repli- 
cation on the confounded treatment comparisons nnd (2) in some ca.«cs, a 
greater complexity in calculations. ... No interaction should lie tomphltly 
confounded unless there la good reason to believe, either from previous ex- 
perience or from the nature of the factors to be tested, tb;it the interaction 
will be negligible. [Cochran awl Cox (1957, p. 212)1 

There is a wide variety of other confoundctl flesign#, of tvhic)i only 
one of the more common ia presented licre. 

Split-plot designs 

If the units which are assigned randomly to tlie various levels of 
one factor (say, Oi, a*, • • • } can be divided into n parts (called plots) , 
then the parts in each unit can be assigned to n levels of a second fac« 
tor (6j, bs, * - *, h») at random. For example, where m =* 4 and n = 2, 
such a design with three replications »» shown in Table 10.11. Tlie 
difference between this and a randomitcd-block design lies in the fact 
that in the latter each of the ten combinations of treatments Is as- 
signed to each of the ten subplots in o replication completely at ran- 
dom. Here the assignment is more deliberate; the randomness of 
the assignment b constrained. 

It the sub-units are regarded ss the expcrimcni.al units, it is seen that the 
treatments [oj, a^, 04] arc applied to groups or blocks of two units. 

Differences among these blocks are confounded with differcncce among the 
levels of A] i.e., the nwin effects of A arc confounded. [Cochran and Cox 
(1957, p. 294)3 

As a consequence, in the split-level design the B and AB (interac- 
tion) effects can be estimated more precisely than the A effects. The 
overall-treatment comparison is the same for this design and for ran- 
domiied block design; therefore it eacrifices precision in estimates of 
A effects. 

... the chief practical advantage of the split-plot arrangement is that 
it enables factors that require relatively large amounts of material and fae- 


Table 10.11. Examfle or a Spur-Ptor Desiox 


Replication 1 RepUcaUon 2 Replication 3 
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tors tfiat require only mull smounts to be combmctl m the same experiment. 
If the experiment js planned to inmtlgate the first typo of heioT, so that 
larRC amounts of matenal are going to be used anyway, factors of the second 
type can often ^ inefuded at very little extra cost, and some additional 
information obtained very cheaply. [Cochran and Cox (1937, p. 297)] 


Qualitative Variables 

Since each variable in an experimental <(estgn is broken into cl.issc?. 
even variables which are ()uantitntivcly defined are, in a sense, treated 
qualitatively. Qualitative variables in the usual sense can also be 
used in cxfierimental designs. For cxanififfl, A may represent color, 
and oi, Oi, may be red, yellow, and so on. It is necessary to be 
sure that different values of a u'ell-dcfined variable (whether quanti- 
tative or qualitative) are used. Suppose that we want to determine 
wfiedier teachers affect stmlent performance. It would be necessary 
to test di^erent teachers. To assure that tbc teachers selected are 
different, tlic »ca1c(s) along which the difference can be established 
must bo made explicit. In such cases it is common practice to draw 
8 sample of teachers an<l atsume timt they are different. Sampling 
cannot assure this difference; lienee, the teachers may be alike with 
respect to the critical variable. If the analysis of variance then 
showed that tJicre was no “teacher effect,” it would not be valid to 
apply this conclusion to the population of teachers from which tlie 
sample is drawn. 

It should also be noted that conclusions based on the analysis of 
variance are applicable only to the range of values of the controlled 
variables which are used. If, for example, temperature is employed 
as a controlled variable, and only values between 40* F and 100“ F 
are used, conclusions dranm from on anaiysls of the results can be 
applied only to the range of values tested. 

For detailed discu.ssioDS of cxperinicnlal designs and the analysis 
of variance see Cochran and Cox (1937), Edwards (1950), Fisher 
(19JS and 1949), Kemplliomc (1032), and Mann (1919). 


REGItESSION ANALYSIS 

Mill’s fifth canon of Induction & caffisf the mel^oi of cancomilant 
vonatiowi. It roads as follows; 

Ifhatei'cr phenomenon varies in any manner whenever another phenome- 
non varies in rome particular manner, is either a tause or an effect of that 
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phenoroenon, or is connected with it throiigh tome fact of causation (1862: 
III, VIII, Paragraph 6) 

Mill's rule is completely qualitative. It was Karl Pearson who pro- 
vided a measure of the correlation between variables. Correlation 
analysis is much used and much abused in science. Of those who use 
it too few understand what a correlation cocfScient represents and how 
it can properly be interpreted. 

In order to understand correlation analysis some attention must 
first be given to regression analysis. Suppose that we are concerned 
with two variables, X and Y. Situations which are characterized by 
any specific value of X, x, may also be characterized by a large number 
of values of Y which are distributed with mean Now suppose that 
the mean values of Y were dclerrained for each value of X. A plot 
of the means would yield a continuous curve, a fegreasion curve of 
y on .y. If this curve is a straight line, w'e have a regression line. 

Let the X-va!ues be represented as deviations from their mean, /»,; 
that is, as (r — ji,), and let represent the mean of the distribution 
means, Then a regression fine can be described by an equation 
of the following form: 


Uvt 


• *1, -h B„{,x — #»,), 


where By,, the regression coefficient, represents the slope of the line. 
This is shown schematically in Figure 10.1. Note that the regression 
line passes through the point ^). Tliis follows from the linear- 
regression equation, since, when (x — /»,) « 0, ft,, = 

If B > 0, the line slopes up to the right; if B < 0, it slopes down to 
the right; and if B = 0, it is horizontal. In the last case is inde- 
pendent of X and, hence, X has no effect on Y. From knowledge that 
B 9^0 we cannot infer that changes in X produce changes in Y; addi- 
tional information is required which we shall consider below. 

In practice we cannot observe all values of X and F. As a con«equence 
we must select a sample of X-values at which we can obser\’e a sample 
of F-values. Then we can estimate $ty by y and v-z by i and B„j by 
where 

_ Si®. 

Sxjy, 


(2x0* 


(18) 


The estimated regression line is 


, = y + hgjjc, - £). 


(19) 
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TJieline wliicb is described bj* thtsetiualion has tbefoUc«rin6intport.tnt 
property: tie sum of the squared vertical deviations o! the observed 
r-valuw from the line {2(y, — j),)) is less thau the corresponding sum 
for any other straight line. Hence, it « called a bost-sguarcs/t. 

Iq practice wc do not know that the regression is linear, but if it is, 
equation (19) estimates the line for us. Even if the true regression 
coefficient is equal to zero {B,t « 0), the estim.ated coefficient by, is 
not likely to be zero because of random fluctuations in the observa* 
tions. Hence, it is necessary to test die hypothesis //a; B,, « 0, using 
the estimate and (usually) a (•test. 

Two important generalizations of regression analysis should be 
noted. First, 1 -values may be observed in situations described by 
values of two variables. In such * case a regression plane can be de- 
rived. Any number of independent %’arisbles may be involved; if 
one or more are, the procedure is called mvltiple-regrestion anaiysh. 

Second, straight lines need not be osswned. Procedures are avail* 
able for cundlinear regression ns well. [Sec, for example, Dixon and 
Massey (1051, pp. lGO-162).) 

It must be rc-erOphasized that A' cannot be said to produce 1' 
solely on the basis of a statistically aigni&cant regression coefficient 
or a favorable analysis of variance. Even if we know that the condi- 
tion ol precedence U salisSed, a Jngnwion etjuslion at best tells us 
how one variable affects another if it affects it, not that if affects it. 
To establish a causal connection we need to know either that the 
various AT-valucs occur in situations which are otherwise tfic same 
with regard to ah other relovant variables (d la Mill's mctho<l of 
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nRrccmcnt), or that the other variables are changing in the way as- 
sumed in nn analysis of variance 

No amount of ohservation alone can nt*uTC us that these conditions 
are satisfied. Assumptions arc always involved in regression analysis 
and the analysis of variance, anti these arc p-art of the experimenter’s 
conception (i.c., model) of the phenomena under study. Strictly 
speaking, then, causc-cffect and producer-product arc rclalionshtps 
asserted by a model, not found in data. The observations may lead 
to acceptance, revision, or complete tcjection of the model. But 
M'itliout some kind of motlel of the situation under study we cannot 
formulate a relevant experiment or dala-collection procedure. There- 
fore, in order to establish the relevance of variables (i.e., their causa! 
connection with nn outcome) we must combine a model of the pos- 
sible relationship between tiic variable and outcome with obscn’ation*. 
The more carefully designed and controlled the experiment or data 
collection, the better is tlic quality of the check. 


CORREL.\TION ANALYSIS 

In some instances we ore interested in two (or more) variables 
neither (or none) of which is subject to our control. In a sample of 
situations in wkich we have no control over the values of the vari- 
ables, we can obsen-c the values of botli. Since neither of the variables 
can be controlled, however, the effect of changes in either on the other 
cannot be determined in that obscr%’otional context, I^t X and V 
represent these variables. 

The regression of I* on A' yields a coefficient 27,,. This regression 
line, it will be recalled, miniraites the sum of the squared vertical 
deviations (if X is plotted on the abscissa). Similarly the regression 
of X on y, and D^, are theoretically obtainable. This line minimizes 
the sum of the squared horizontal deviations. 

Suppose that X and Y are independent and that their values arc 
normally distributed. Then vertical distances can be expressed as 
deviations from the mean, tv We know that, if X and Y are inde- 
pendent, a horizontal line through ftp minimizes the sum of the squared 
(vertical) deviations of the y’e. Correspondingly, the regression of 
X on 1" in this ease will he a verUcal line drawn through The 
angle between these regression lines Is 90'*. This is shown in Figure 
10 . 2 . 

Now consider the other extreme in whicdi X and F are two ways of 
characterizing the same property. For example, X may be length in 
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Ficiuhr 10^ Lbcar rrp'e«fion of intir- 
pcadsnt variabJrs. 



!cet, and 1' Irnglh in inebes. (Soc FigDrc 10.3.) AJ] observations 
will lie on a straight line described by the equation 

Y » 12.V or A* - ^Y. 

Tliis Jinc nttoimizes the sum of both the squared vertical and iioH* 
zontal observations. The angle between the two identical lines is 0*. 

The angle between the two regressions can be used to measure the 
degree of association between values of the two variables. The larger 
this angle^ the less the association; hence it is conrcnlcnt to use the 
cosine of the angle, which is 0 for 90^ and 1 for 0^. This cosine is 
called the correlation coej^ctenf (f), where 



Then, clearly, if both regression coefficients are rcro, as they were in 
the first case, #>= 0; the variables arc uncorrclatcd. In the second 



Flaua tOX liegrc^on hetwteB e<]uir* 
stent vnrishlcs. 
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case, pyt = 12 and Pxv = Ma- Tl'® square root of their product is, of 
course, 1. 

Regression coefficients are negative when the dependent variable 
decreases as the independent variable increases. If two variables 
each have negative coefficients relarive to the other, then the angle 
between them is greater than 90® and hence the correlation coefficient 
is negative. At the extreme the lines overlap on an angle of 180 
whose cosine is —1. Thus correlation coefficients vary between —1 
and +1. 

There are several common misuses of correlation against which the 
researcher should be on guard. First, an estimated correlation coef- 
ficient will seldom be zero. This docs not necessarily mean that the 
true coefficient is unlikely to be zero. Consequently, the hypothesis 
f/o! p = 0 eliould always be tested to determine whether or not the 
correlation estimated from a sample can be considered significant. 

Second, correlation analysis presupposes a random sample. When 
applied to data collected in another way, it may not provide a basis for 
inferences to the population. 

Finally, a significant correlation between variables does not estab- 
lish either as the producer of the other. If two variables are not 
correlated, however, we can conclude that they are not causally re- 
lated. Thus correlation can play an important role in science: the 
elimination of irrelevant variables. 

Correlation has another important use in prediction. The values 
of one of the two correlated variables may be quite easy to determine, 
whereas the other may be difficult to determine or else cannot be 
determined until it is "too late." For example, suppose we know that 
income and education are positively correlated, and we want to 
sample a bigh-income group. If it is difficult to determine income 
but easy to determine edueation, we could select a high-education 
group. 

Most economic predictors are based on correlation. In preparing 
sucli predictions one must be sure that the variables to be used as a 
basis of prediction are known before the variable to be predicted. 

It should also be noted that one is not restricted to two variables 
in correlation analysis. Any finite number can be treated. Also, the 
eoiTcIfltions need not be linear; rarvilinear procedures are available. 
For detailed discussion of correlation analysis see Ezekiel and Fox 
(1959), Johnson (1949), and Kendall (1948). 
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CO^tBINING THE ANALYSIS OP VARIANCE 
AND BECRESSION ANALYSIS 

The analysis of variance and regresrion analysis can be combined 
in two different ways. In the first the analysis o! variance is ap- 
plied to data obtained by means of an experimental design in order 
to determine whether the "experimental variable" has a significant 
effect on the outcome under study. 1/ it does, tJje same data can 
then be subjected to a regresaon analysis. The following example, 
taken from Cochran and Cox {lOSO, pp. 00-97) , illustrates this combi- 
nation of procedures. 

A randomiicd block design involving three replications was used to lest 
the effect of potash (K«Ot on the yield and properties of cotton. Five 
levels of appI5c.stion were used; 35, 51, 72, IW, and 144 ponada per acre. A 
single attople of cotton was taken fr^ each plot and four detemurmtiocs 
of the breaking strength of fibre were made on each sample. The mcaas of 
the four observations are shon-n in Table [10.12]. 

Tiie anolysis of variance indicated that strength of cotton is sig- 
nificantly affected (at the 6 per cent level) by tlie amount of K;0 per 
acre. The data also indicate t)mt the strength increases as tl^e amount 
of KgO decreases. Consequently a regression analysis of strength on 
amount of ICjO waa perromed. It yielded an estimated regression 
coefficient equal to 0,0050, whicb is significant at the I per cent level. 
Hence one can conclude that untkin the range tetted increased appli- 
cation of KjO produces a weaker fiber, the strength index declining 
by (18) (O.OOoO) or 0.090 for each 18-lb increment in KbO. 


Table 10.12. SniEwmi Ikdcx or Cottos in a 
IlAXDOUtZEU RlOCK ExpEniUENT 
Treatment Replications 

(pounds KiO ■ — ' 

per acre) 1 S 3 Totals 


30 7,62 8.00 7.93 23.55 

54 $.14 8.15 7.B7 24.16 

TZ r.K r,TS T.T-t S3.23 

108 7.17 7£7 7.S0 23.54 

144 7.45 7Je» 7J1 22.35 

Totals 3$.15 30.13 3S.55 115.83 
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The second way of combining regression Analysis and the analysis 
of variance yields an analysu of covariance. 


THE ANALYSIS OF COVARIANCE 

R. A. Fisher is also largely resfiORsible lor on important extension 
of the analysis of variance, the onofym of covariance. This procedure 
is applicable to situations in wliicfi a vnriafilc alTccts the ohscrN'cd 
phenomena but the value of this variable cannot be controlled during 
the experiment; that is, it cannot be incorporated into the design. 
For example, suppose that “miles traveled” conditions the performance 
of automobiles, but that in drawing a sample of automobiles the value 
of this variable cannot be determined In advance. Then, by the use 
of the analysis of covariance, the observed performances (F) of the 
automobiles can he adjusted for the effect of "miles traveled” so that 
the effect of the controlled variables can be determined. 

In some cases the researcher may discover a relevant variable only 
after the experiment has begun. During the course of the research, 
fop example, it may become apparent that Uic performances of the 
cars are affected by how frcnucntly they have been lubricated. If 
we can determine this frequency, even after the experiment has been 
conducted, then by use of the analysis of covariance the observed 
perfonsances can bo adjusted for this experimentally uncontrolled 
source of variation. 

The logic of the procedure can be understood by reference to a 
simple example. Suppose that we want to determine whether values 
of a property of a phenomenon (V) change when values of a control- 
lable variable A are changed. Suppose that there is another variable. 
B, which is suspected of having an effect on V, but values of B cannot 
he controlled by tlie experimenter. Tlie values of A, however, can be 
established by the experimenter for each observation. The problem, 
then, is to cancel out the effect of fi on F so that the significance 
of the independent effect of A on F can be determined. 

Assume that A is divided into three intervals: m, Oj, and 03 . Assume 
also that four observations of V are made for each value of a and that 
the B-value of each observation can be determined at the time of ob- 
servation. The data obtained may be presented assbown in Table 10,13. 
These data can be graphed (see Figure 10.4) in such a way as to let the 
abscissa represent ^-mlues, the ordinate F-values, and the point of 
origin the average observed B-value, B, and the average obse^^'ed F- 
value, F. 



EXPERtSIEJfTATtOS' AST) COfUm-ATinS 337 


Table 1013, Data por AsiHstH op Covabusce 

Ox oi at 

n V n r o v 

2 S 4 10 0 c 

1 7 3 11 2 5 

0 7 ) 6 4 H 

— - — TobJ ToLiJ 

1 10 4 13 2 5 /I r 

Tolrtls 4 32 12 40 8 24 21 00 

Mcnat 1 $ 3 to 2 C 2 8 

1 } r 


Similar ecparAtc pidts can Im* n)n<lp for the Uittn in columns Qi, as, 
and aj, as is done in Figure 10.6. 

The throe plots iliown in Figure 10.5 can be superimposed on one 
another to obtain Figure lO.C. Ry so doing, the mean* of each /!• 
column has been stipcrimposed, nod similarly for the t ’-columns. Ke* 
the regression line of V on C can be determined for Figures 10 4 and 
10.0. The estimated re>;TCSsion cocIBcimt of the former (br) is ap- 
plicable to the obser\'nlioni taken ns r whofe. Tlic estimated regres- 
sion coclJicient of the latter (h») is U>c •aithin-group rpgrr««ion, since 
the plot in Figure 10.6 has cancclwl out between-group variance. 




Adjusted V-values 
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against B-values 


15 

10 



£-value« 

Plot of y-values (for 02 ) against £-values 



plot of V-» 3 l«es (for 03 ) against B-values 
rjcciiB 105 Column plots of l*-valuM against B-\alues 



Fioues 105. Plot of adjusted 
P-vslues against adjusted 
values. 
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If the dispersion niwiit the regmsion line in Figure lO.C is signifi- 
eanlly less than in Figure 10.4, equating Uie means of the subgroups 
had a significant ofTcct on the data From this it ran be inferred that 
the means of columns a,, oj, anil oj arc significantly dilTcrcnt in the 
original data and hence A has an rJIect on Y. T]ie analysis of vari* 
ance can be applied to the data, which liave liccn adjusted in tiic way 
the data in Figure 10,0 have been. If the ciiange from Oi to dj to a* 
)).ad no c/Tcct on I’, then equating iJieir means should not effect the 
dispersion around the regression line. 


MCniOD OF RESIDUES 

The atie canon formulated by Mill which has not yet been con- 
sidered is called the method oj residues. It rc.sds os follows: 

Subduct from any phenomenon such part as Is known by prc^^ous in- 
ductions to l>c the ciT<'ct of certain antce^ents, and the residue of the pbe- 
Domroon is the effect of the remamiog sntcredcDts (15£9, III, VlIJ, Fars- 
grapU 5) . 

An exam]de of this method occurs in the experiment of Madam Curie 
in which she discovered radium. She obtained some pitchblende from 
the mines of Junchiinslhal in Ctcchostovnhia. Tliis ore contained 70 
to 75 per cent of black uranium oxide. On examining tlic ore, she 
found ft to be tnucii more radioactive Uian other ores of Uie same 
uranium content. She came to suspect tliat Uie ore contained a radio- 
active subelanco other than those with whicii slie was familiar. She 
eventually rcinoveil all t!ie known substances and isolated radium 
and its effect. 

AnoUicr example of the mclhoil is to be found in the didcovery of 
the planet I’luto. Fcrcival Lowell, after accounting for all the known 
influences on the planet Uranus, still had an unexplained residual 
effect on its motion. Tlio ousteDcc of another planet was assumed. 
Its size and location were estimated on tlic basis of the residual effect 
to be e.\plrtined. On January S3, 1930, C. IV. Tomaugh discovered 
the planet Pluto, which satisfied these conditions. 

It should be noted that tliis method presupposes that certain causes 
and effects are already known, presumably established by other 
methods. Strictly speaking, this method is a variation of the method 
of agreement. What wo have is a set of "antecedents," say A, D, and 
C; and a phenomenon consisting of a set of effects, say X, Y, and 2. 
If C is removed, say Z is also, and similarly for B and Y. Then A 
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is said to be the cause of A‘ The presence of the effects Y and Z is 
really incidental- What we have is 

ABC followed by X 
AB'C' followed by X, 

which is the method of agreement. The difference lies in the fact 
that in the method of agreement tl>e cause is identified in advance, 
whereas it is "discovered” in the method of residues. Nevertlieless, 
the earlier discussion of the inadequacy of the method of agreement 
as formulated by Mill is relevant with rcsj>ect to this method. 


CONCLUSION 

The relative advantages and disadvantages of different experimental 
designs generally must be expressed in qualitative terms. Quantita' 
tive comparisons between some characteristics of two or more designs 
can be made in eome instances, but the criterion employed is statistical 
rather than pragmatic (e.g., minimum variance of estimates). Be- 
cause wc cannot yet (1) characterise all the possible experimental 
designs along quantitative scales and (2) generate cost-of-error func- 
tions, comparisons must be made in spccifsc contexts rather than by 
use of analytic optimizing procedures. 

To generate a cost-of-error function for experimentation it is necct- 
sary to have a decision model, but it is precisely in order to formulate 
such a model that experimentation may be required. As yet we have 
no way of escaping this vicious circularity, as we do in estimation 
theory. This is not to say that the difficulty may not be overcome 
eventually. 

For the reasons considered, experience remains the best available 
guide for the selection of experimental designs. This makes the de- 
sign expert an invaluable asset in conducting any research requiring 
experimentation. 
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11 

DERIVING SOLUTIONS 
FROM MODELS 


INTRODUCTION 

The purpose of coostnicltog a model of a problem situsUon is to 
enable U8 to determine what values of the controllable variables (X’d 
provide the best measure of performance (V) under conditions de» 
scribed by the parameters (!/>. Tlicrcfore, once we have a model of 
the form 

V«/(.Y/.V,). (J) 

we w’ant to determine what values of A'i either maximize V (if it is a 
measure of desirable performance) or minimize it (if it is a measure 
of undesirable performance). The solution of a problem, then, con* 
gists of a set of equations, one for each controllable variable, of the 
form 

(2) 

where the values of Xt thus specified maximize or minimize V. In this 
chapter we consider procedures by which optimizing equations of tlie 
form shown in (2) can be obtained 

The procedures we will consider are not restricted to problem 
solving. They can be used also in answering questions involving the 
maximization or minimization (or, more generally, the evaluation) 
of any dependent variable which is ejgjressed as a function of a set of 
one or more independent variables and constants. 

3(2 
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Among the pmcedurcs we wifi consider ore several which have uses 
in research other than in deriving solutions from models. At the risk 
of disrupting the main line of this discussion these other uses will be 
diseusscil because they arc important. 

It should be kept in mmd that we arc concerned with deriving 
solutions from models The value of the solution depends on how 
adequately tlie model represents the problem situation; the ndequaej- 
of (he solution depends on the adr4)uary of tlic model. Testing for 
such adequacy is the subject of Cliapter 13. 


TYPES OF DEnn’ATIONS OF SOLUTIONS 

Consider the clcmenlarj- mnlel that was n«td in fh.nplers 7 and 8: 

v-.vr + i. C3) 

ivherc A’^O In geoeral, there arc t«-o waye of going about solving 
the model. In tlie first we proceed deductively, by onolysts. lo the 
second we proceed inductively; (hat is, numenfolly. To solve the 
model Hnalylically we proceed as follows: 



Ifcrc we have used elementary calculus to find the value of X whicli 
minimizes V.* The solution obtained is completely general; it holds 
for any value of T. It is also completely abstract, since it did not 
require knowledge of the value of Y to obtain it. 

It we know the value of Y, say 4, Uicn we can obtain the optimal 
value of X from equation (5), %. But, if we know that T is equal 
to 4, wc enn obtain the solution by substituting values of X in equa- 
tion (3) and by trial and error find that one which minimizes V. 
Graphical techniques may be used to assist in this process. The first 
procedure, whicii uses equation (fi), consists of a numerical evalu.ation 
of a solution that has been obtained aaab'Ucally. In the second pro- 

•Thc conditions which sasurc this bring a minimani tsnd not a miiimum) 
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cedure, wljieli uses equation (3), the solution itscU has been directly 
obtained in numerical (rather than abstract) lorm. 

In analytic procedures we do not consider any particular set of 
values of the controllable variahlcs; we proceed directly to the solu* 
t'lon. In numerical procedures xve tty out various values of the con- 
trollable variables and select those that yield the best result. For 
rather obvious reasons analytic procedures arc generally preferred 
where they can be used, but there arc many eases where they arc 
cither impossible or impracUcal. In many of these, numerical pro- 
cedures will yield an exact or approximate solution. 

No effort will be made here to survey the large number of mathe- 
matical techniques which can be used for deriving solutions from 
models. Familiarity with their use or availability should be ex- 
pected of every researcher. Consequently, after a few general re- 
marks about the more common of these lechniquca we will devote 
this discussion to more recent dc%-clopmcnls about which knowledge 
is not yet generally spread among researchers. 

We will not discuss (he methodological problem associated with 
selection of the best analytic or numerical procedure for deriving a 
solution from a mode). In most eases the best choice is relatively 
apparent to one who is familiar with the alternatives. He must 
balance tbe cost of using the technique against the cost of error which 
it may yield. This is particularly Important where approximation 
techniques ate concerned. Furthermore, even if we wanted to, we 
cannot j'ct represent the choices io a research decision model. 


ANALTTIC PROCEDURES 

In equations which contain only one controllable variable, its 
optimizing value, if there is one and it is obtainable by analysis, can 
be determined by tbe use of the calculus, as was done in equation (4). 
The derivative of the outcome variable is takfm with respect to the 
controllable variable, set equal to zero, and solved. We must take 
tbe necessary steps to detennine whether the function is at a minimum 
or maximum point where the derivative is equal to zero, and whether 
there is more than one maximum or minimum. 

If there is more tlian one controllable variable, the partial derivative 
of the outcome is taken with respect to each controllable variable and 
set equal to zero. Then, under certain conditions, solution of the re- 
sulting set of simultaneous equatirais yields tbe optimizing values of 
the controllable variables. 
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If tlic VfiluM of Iho conlroIIaWc variaWcs are constrained (re- 
fUictcil) by one or more vnuations or in«nialions, and if there arc 
ojjjy a few such constwintj, wc can u«c I/agrafiKisn muJtipIiers or 
eomo variation thereof. Wlwn the tuitnlicr of con?lrainls becomes 
large, numerical procedurea nrc generally rcriutred, Here tlie rc- 
rently iJcvrJo/xnJ techniques of niaUu'niatiea! fjrogramming arc par- 
ticularly useful. 

Finally, in roine case* an optiiniiing value of a function rather than 
n variflbli) miMt be found. In principle Ihe rnlciilus of variation* i*< 
npplicalle to such fa*c», hut in imicticc «c e.-m nelualty eolve only 
very few problems uriiig this technique Normally we must resort 
to numerical techniques, p.irticulariy fli« technique of dj-namic pro- 
gramming. 


NUMEIllCAL rROCEDUnnS 

The lamiliar triabnod-error technique has already bi'ca refpnrd 
to &8 a numerical proce^Iure. Frequently, with the help of gfaphie 
pfots ono can more rapidly determine the neighborhood of the opti- 
mum and concentrate one’s trials in that region. Since trial and error 
can be very time coinuming. effort* have l>ecn tn.vlo to modify these 
proeedurw to that they converge on Uic eofuUon. A numerical pro- 
cedura which tends toward a solution on «iiccmi\T trial* U called 
I'fcrcfiVc. Tlie mathematical programming techniques to which rel- 
trence fias been made nrc alf iterative. 

The Jai^c amount of compubirion requite*! even by c/Rctcnl Iterative 
proccdunrs prohihUcil llieir c.xlcmive application until the advent of 
the electronic computer. W'itli (he development of eoruputers and 
techniques for programming them, the u'C of nuTnericnl procedures has 
Ucome much more widespread. 

Kwentfy Uicrc has liccn another development nhich may mate 
numerie.'il proecdurea even more accessible. It Involves the use of 
logical trchtiiques to re*lucc (he mintltcr of povriblo solutions which 
must be eonsitlcred. For exaroj^, eonshJer a problem in widcli « 
tasks must t>c porformetl, each requiring u.«e of n number of different 
machines, where only one of each type i* available. Which task should 
bo performed first on each roarhijw so iJiat the total time required to 
complete the n tasks i** minimised? If tlicre nre in machines involved 
there are {«/)“ iwasiblo eolutions. For m and n of any size it would 
Iw much too time consuming to ti>' all of Oiese, Akers and Friedman 
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(1055) have shown how the class calculus (the Boolean algebra) 
ean be used to reduce the cases in which there arc 2" possible solu- 
tions to a relatively small mirnlwr of fca«il)le Folutions which satisfy 
the conditions of optimality. Then only thc'-c iicc<l Ik* cvahiatcil 
numerically. 

Such instances of the use of logic to reduce euinpulntiun are stilt 
rare, but they hold promise for the future. 

Among the important developments in iiuiiicrical analysis stimu- 
lated by the advent of computers one of the most (irnmatic is Jt'mu- 
lation. It is to this procedure that most of our attention will l>c given 
in this chapter. 


SIMULATION 

With the advent of the computer, simulation has come to attract 
increasing attention in research circles. It is wrong, however, to 
associate simulation exclusively with computers because their use is 
not essential to tlie procedure. Nevertheless, computers have made 
simulation possible or practical in many rcscarclj situations in which 
previously it was neither. 

Simulation has come to mean di^erent things to different people, 
depending on the use to which they put it. It has been utilized for 
personnel selection and training, and even for therapeutic purposes. 
Our concern here, however, is only with its application to problem 
solving. It has another important research use which will be dis- 
cussed in Chapter 14: to display rcfcarch results to those who must 
be “sold” on these results. 

It is much easier to characterize simulation in figurative terms 
than in literal terms. For example, an essential characteristjc of 
simulation is captured by the observation that a model represents a 
phenomenon, but that simulation imitates it. Tliis dynamic aspect of 
simulation is also caught in the figurative observation that models 
are photographs and simulations are motion pictures. These figures of 
speech are not meant to imply that motlels and simulation are to be 
contrasted, because simulation is a tcay of using a model. It is, in 
effect, experimentation on a model rather than on the j)henomenon 
itself; that is, it is vicarious experimentation. 

In principle, everything that can be accomplished by simulation 
can be accomplished by experimenting directly on the phenomena 
involved in the problem. In practice, however, it may be impossible 
or impractical to experiment on the phenomena themselves. 
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Tlie conditions under which simulation is useful arc summarized 
hy Jfof^enthaler (1901, pp. 366-367) as follows: 

It may not be possible to ol«erve the phenomenon in its desired environ- 
ment. This is true of studies of the thrust of rocket motors for use in inter- 
pinnefary space. The phenomenon or sj'stcm may be loo comples to sum- 
marize m a compressed mathematical fonnulation. It has not been possible 
thus far, for example, to reduce the operation of a large bus-mes activity to 
a few simple equations. Analytical techniquos may not exist for solving the 
m.atlicmatical formnbuon once it has been aehiev^. This is often the case 
in solving the ihfftismn equation or other p-arttal-differcntial equations when 
the boundaiy conditions are lime-dcqtendent or complicated by other demands 
of realism. Even when analists have the confidence and ability to arrive at 
a theoretical prediction of the behavior of a large system, it m.iy not be 
pa'sible to perform validating experiments You cannot, for example, test 
eoneJusions about global strategic ir.ar by iT^inn them even once. IVfien 
any of these dijfievUits occur, as thq/ do dady tn the attempts . . . to taeUe 
previously untotirhed, unmanageable problems, some form of nmulalion is the 
ohiaous tool to be tried. 

The principal use of siroujalion in rJeriving or testing solutions 
from decision models occurs when the measure of performance is a 
statistical parameter of a distribution of outcome. In many such 
cases the optimiziog values of the controlled variables cannot be 
derived from the mode! bj* cither abstract or (the usual type of) 
numerical analysis. Simulation involves the use of cumbers and 
hence is n typo of numerical analysis or, more properly speaking, is 
an extension of it. In numerical analysis the numerical values of 
the controlled and uncontrolled variables are inserted in the mode! 
and the outcome is calculated by norma! aritlimetical operations. 
By trying a number of possible solutions the best (or approximately 
best) one can be identified and selected. In simulation we also gen- 
erally try a number of solutions, but tlie di/Tcrenee from norm.xI 
numerical procedures lies in the way each solution is evaluated. 

The evaluation or testing of a proposed solution to a decision 
model by simulation consists of running the system on paper or in 
a computer (i.e., symbolically) for a set of values of the controlled 
variables in order to generate enough instances of outcomes so that 
their distribution can be determined. From tiieso "observations” the 
necessary parameter is estimatwl For example, in a "wniting-line” 
problem it may not be possible to delermiDe analytically how m.sny 
serving facffffies one oc^if to have sa as to minimiee Uie total cost 
of operation. But for any spccifial number of facilities we may be 
able to run the operation on paper by "imitating" arrival of cus- 
tomers, their waiting, their scleciion for service, their servicing, and 
their departure. dJy observbig these operations we can estimate the 
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distribution of outcomes (e g., the daily cost of operation) and calcu- 
late the value of a measure of performance based on it. The Bame 
holds true for other types of phenomena: tlie behavior of small physi- 
cal particles, cells reproducing, the spreading of a disease, people inter- 
acting, and societies fighting a peace or a war. 

The various uses of simulation in problem solving are best under- 
stood with respect to the familiar form of the decision model: V ~ 
/(A’l, V,). The four principal uses arc as follows: 

(1) To determine the optimizing values 0 } controlled variables. 
For example, in queuing theory <the stochastic Ihcoiy of waiting-line 
processes) models expressing such outcome variables as “mean wait- 
ing time” or “mean length of queue” can be expressed as functions of 
the distributions of arrivals and servicing times, the number of servic- 
ing points, the order in which items arc served, and so on. But the 
effect of the number of servicing points on the outcome can be deter- 
mined analytically only for certain distributions of arrival and ser- 
vicing times fe.g., Poisson and exponential). For some distributions, 
the necessary integration cannot be performed. But by simulation in 
these cases, outcomes can be estimated, and hence solutions to prob- 
lems involving queuing processes of this type can be approximated. 

(2) To study transitional processes. In many cases where a model 
can be solved analytically, the solution specifics only the terminal or 
steady state that results from changing the valuea of the variables, 
and not the intermediate stales, the slates of transition. Simulation 
exposes the transition to as careful a study as the researcher may care 
to make. For example, the solution of a complex inventory problem 
involving the purchasing, storage, and use of a large number of items 
(c g., spare parts for aircraft) may show that the current stock levels 
of some items are too high and of others arc loo low. The solution 
obtained analytically may lelj us what the average inventory invest- 
ment will be after the changes have been made and after the sys- 
tem has settled donm to a steady stale. In getting to the inventory's 
steady stale from Us current state, items which are understocked ac- 
cording to the solution can usually be brought up to the level indicated 
by the solution rather quickly by buying more of them. But items 
which are overstocked will be reduced to the appropriate level only 
with use, and this may take some time. Consequently, although the 
inventory investment may eventually decrease if the solution is fol- 
lowed, it will usually increase during the transition. It may be im- 
portant to know by how much it will increase and how long the 
period will be before the steady state is reached. Simulation makes 
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it possible for us lo "map” this transition anil determine its charac- 
teristics. 

(3) To eslimatc values oj model parameters or the model's func- 
tional form. In some cases we may be able to construct a model but 
not lo evaluate alj its parameters (Yf) because of the lack ol data. 
We may, liowever, have good end plentiful data on past outcomes and 
values of the controlled variables. In such cases we can by simulation 
try out a large number of possible values of the parameters, together 
with known past values of the controllable variables, until we obtain 
one or more seta of values which yield outcomes that correspond 
well with the known past outcomes. The same kind of procedure can 
be used lo explore alternative functional forms of the model. 

(4) To treat courses of action which cannot be formulated into the 
model. In some problems the performance of an entity under a set 
of specified conditions may be one of the important variables, but 
we may not be capable of enumerating or char.'ictcrising all its pos- 
sible courses of action in advance. Even if wc can enumerate them, 
wo may sot be able to characterize them by a set of quantitative 
variables. Such a problem is quite common when sucli an entity is a 
decision maker and when the conditions involve other (cooperating 
or competing) decision makers. iMien such an entity’s performance 
cannot be modeled, the entity itself may bo put into a modeled situa- 
tion to determine the effects of Its behavior as well ns of other vari- 
ables on outcomes. When this entity is humen, such simulation is 
cailed paminff. The best-known examples of such simulation are mili- 
tary games, where real and modeled elements arc combined to pro- 
vide a basis either for prcdictiog the effects of such variables as num- 
ber and types of forces and equipment of friend and/or enemy, ter- 
rain, and weather; or for determining what slratcgics and tactics (con- 
trollable variables when considered relative to "our” side) are avail- 
able for choice. 

Types of Simulation 

Simulation procedures can be classified into three main types, de- 
pending on the kind of model which is used: iconic, analogue, or sym- 
bolic. Since models may have a mixture of these charaeterislics, so 
may simulations. The three types may be described as follows. 

Iconic simulation 

Iconic simulation is the manipulation of an iconic model under real 
or iconically represented conditions. (An iconic model, it will be re- 
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called, has the same essential properties as that which it represents, 
but with a transfonnation of scale.) Such simulation is widely used 
in problems involving the construction or production of an object, in 
so-called hardware or design problems. For example, the testing of 
a small physical model of an aircraft in a wind tunnel is an iconic 
simulation. Similarly, the testing of ship models in tow tanks is also 
an iconic simulation. The construction and operation of pilot plants, 
a common procedure in the chemical industry, constitute another 
example. 

Analogue simulation 

Tills type of simulation involves the manipulation of an analogue 
model. For example, the Xfoniac, a hydraulic model of the British 
economy constructed at the I-omlon School of Economics, can be op- 
erated to estimate tiic effects of certain monetary policies of the gov- 
ernment. The wide use of analogue computers, particularly in the 
design and control of continuous production processes, involves such 
simulation. In the study of (iueuing processes arrivals of “customers" 
at service points have been simulate by devices whicli emit radio- 
active particles at random. 

Symbolic simulation 

Symbolic simulation is a process by which equations arc evaluated 
numerically. The essential characteristic of the process is best de- 
scribed with reference to an equation of (he following form: 

\y^no..Ui), (G) 

where IF is a property of a statistical distribution or a distribution itself. 
Ui represents one or more (controllable or uncontrollable) varia- 
bles which are a property of a distribution or a distribution 
itself. 

Ui repre.«ents other independent variables and constants. 

Symbolic simulaUon consists of selecting n probability sample of val- 
ues of each of the stochastic variables t/| and (relative to fixed values 
of the 17/) calculating a particular value of the outcome w for each 
set of sampled values of the Ui- Then IT is estimated from the re- 
sulting set of te's. Thus, the effect of varying the V/ can be deter- 
mined by repeating this process for different values of them, Tliis 
requires that the equation In the form of (C) be transformed into a 
particularized form: 


w =/•(«.-. t/<). 


(7) 
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where u* represents a set of particular values of the stochastic vari- 
ables Ut obtained by sampling from the distributions of which t/j arc 
parameters or which Ut represent. The function /* is a suitably modi- 
fied form of the general function /. 

Random sampling from the probability-density function lies at the 
heart of symbolic simulation. Tins application of sampling is called 
the Monte Carlo procedure. As wc shall see, this procedure can also 
be used to evaluate nonstatislical expressions and variables. 

Some Simple Examples of the Nfonte Carlo Technique 

In order to provide the reader who is unfamiliar with the Monte 
Carlo technique with some background for the subsequent discussion 
we will consider several examples of its use. These examples will 
show how one can apply the principles of random sampling to the 
evaluation of detenninistic and probabilistic variables. They will also 
make clear that l!ie Monte Carlo procedure yields statistical estimates 
which are subject to error, and hence that characterisation of this 
error is a critical aspect of use of this technique. 

Consider a roulette wheel on which there are no markings except for 
one band indicated along the circumference. Suppose that we want to 
determine the angle whose arc is the marked band, and that no angu- 
lar measuring devices are available. Assuming that the wheel is un- 
biased, wc could proceed in this way. Wc spin the wheel and obser^’e 
whether or not the pointer stops in the marked band. We repeat 
this process a number of times. The proportion of trials in which the 
indicator stopped in the baod is determined. This fraction can then 
be applied to 360” to estimate tlie angle in question. In this way wc 
would have estimated a detenninistic quantity by a sampling proce- 
dure. Tlic larger the number of spins, the smalJer will be the error 
of the resulting estimate, assuming that the wheel is unbiased. 

Now consider the following case, which can be solved either by anal- 
j'bls or by simulation. Suppose that we bare a triable IT, which is the 
mean of a joint distribution of variables tJi and Ug. Assume that Ui 
and Vg arc normally distributed with means I7i =* 10 and Ug *= 25, 
and ffi = 1 and vg = 5. In this case we know that 

TT- Z7, -I- 10 + 25=. 35 

but let us see how we can estimate the value of fT by Monte Carlo 
procedure?. 

In column (1) of Table ll.l isli^ed the nximber of each of 15 trials. 
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Table 11.1. An Examble or a Monte Caklo Estijiate 


0) 

(2) 

(3) 

(4) 

(5) 

(6) 

(7) 

Number 

Random 

Isormal 


Random 

Kormal 



Estimate 

Mean 

{Cumulative! 

1 A 


rj 

Pj + Tlffi 

r» 

Ui + fjS* 

u, 4- «, 

fl? 


1 

0.4M 

10.4M 

0.137 

25.6S7 

36.151 


2 

0.060 

lO.OCO 

-2320 

12370 

22.430 


3 

1.4S0 

ii.m 

-0351 

23.230 

31.716 


4 

1.022 

11.022 

-0.472 

22.640 

33.662 


5 

U04 

11394 

-0355 

22325 

33.619 

32.116 

6 

0.206 

10.900 

-0313 

22.435 

33341 


7 

1.179 

11.179 

-1.055 

19.725 

30.904 


S 

-1.50J 

8.499 

-0.4SS 

22.560 

31.059 


0 

-0.690 

9.310 

0.756 

2S.7S0 

38.090 


10 

1372 

11372 

0.225 

20.125 

37.479 

33.145 

u 

2.455 

12.455 

-0323 

233S5 

35.840 


12 

-0331 

9.469 

-1.940 

15300 

24.769 


13 

-0.631 

9366 

0.C97 

2S.485 

37.851 


U 

1379 

11379 

LC9$ 

33.490 

44.769 


IS 

0.046 

10.016 

0321 

26.605 

36.651 

34.0S9 


In colnmns (2) and (4) tandom normal numbers are listed which have 
been taken from a table of fuch numbeis. Since these numbers are in 
ff-unita, they must be converted into “observed values" as is done in 
columns (3) and (5). We add to the appropriate mean {Ui or Vi) the 
product of the random normal number and the appropriate standard 
de\Tation (<ri or o-j). We then add the “observed ralues” ui and U 2 to 
obtain a value of w, shown in cdumn (6). The averages of the first 5, 
10, and 15 tc’s are shown in column (7). The estimated standard error 
of these estimates may be obtained by usual statistical procedures. 

Design Phases of Monte Carlo Teduugue 

Use of the Monte Carlo tertmique involves three research decisions: 

(1) how to obtain a set of random numbers; 

(2) how to convert these numbers into random variates from some 
specified probability distribution; and 
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(3) Ilow to increase tlie efBclency of wlimatcs obtained from the 
sampling process; Umt is, bow to rcdace the variance of the estimates. 

Random numbers 

TJio basic tool cmplayctl in symbolie rimulation is a set of random 
numbers. The numbers nhich arc employed arc either supplied by 
avaihablc tables or decks of punehetl cards prepared from sucb tables, 
or tJiey are generate*! by some aritlimetical procedure. Numbers gen- 
crnte<l in Uiis way arc referred to as pteudo-random numliera. They 
are used primarily in ."imulation involving a computer. Tlicir princi- 
pal advantage lies in Uic fact that a large portion of the computer's 
memory is not then filled with random numbers; only a generating 
routine is required. In adilillon, with eurb a routine it is easy to 
reproduce the simulation using the same random numbers, thereby 
allowing more direct comparisons between the cfTccLs of different val- 
ues of the manipulated s'ariaWei. The disadvantages arise from the 
possible deviations of pseudo-random numbers from “pure" random- 
ness.* 

The most commonly used method of generating pseudo-random 
nuinbera Is called the mtd-i^uore method. It consists of taking, for 
example, a four-digit number, s.ay 3163, and squaring it to obtain 
10,125,12-t. Then one takes Uie middle four digits, 1251, and squares 
this number, and so on. The four-digit numbers thus obtained con- 
stitute a set of pseudo-random numliers. Tlicse numbers eventually 
come back on themselves; Uiat is, go into a cycle which is generally 
of the order of 10* but may go as high as 10*. By Uie use of oilier 
procedunfs, such as the eongruential melhodt, cycles as large as JO** 
iiave been oblainctl. The congruential metliod consists of letting 

A',+, - A*.Y,(mod if) 

That is, .Y.,+1 l^ equal to the number that remains after A'.Y, is di- 
vided by M. This ixicthod was first reported by Lebmer (1051), 
using K = 23, M = 10* + 1. Tins yielded a sequence of eight-digit 
numbers with a cycle length of 6,862,352. Taussky and Todd [in 
Jfeyer (195G, p. 17)] report that in using A’=s5*’, = 1, and 

^f es 2‘* they get a sequence of approximately 10** numbers before 
cycling. 

Concerning these procedures of generating pseudo-random numbers 
Marshall [in Meyer (1956, p. 9)J olwervcs that; 

•For more eoinpleto dlscuasion thaB U prorWed bets of trstbg and gene- 
ration of random numbera Me Tauasky and Todd in Meyer (tOSO, pp, liff.). 
and article by K. D. Tocher in ‘'Sympoaiiini on Monte Carlo Methods" (19M, 
pp. 29-61). 
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The bogus character of the randomness of the generated sequence does not 
seem to affect the Monte Carlo calcolations, just as it is not detected by the 
various statistical tests to which these sequences have l)ecn subjected, and 
through which they have passed with flying colors. 

Random variates from a specified probability distribution 
Given a table of random numbers or a set which is generated by a 
pseudo process, it is necessary to convert these numbers into sampled 
values from the distribution involved. Tijc general jogte of such 
sampling is quite simple, although the actual operations may be more 
complex. In principle what is involved is the following. Consider 
the distribution in question as being plotted in cumulative form. (See 
Figure 11.1.) The ordinate can then be divided into a large number 
of small equal intervals, say a thousand parts. Then random numbers 
are selected from 1 to KXK), and for each number a horizontal line is 
drawn from the correspondiag point of the ordinate to the cun’e 
represenlmg the distribution. The value of the point on the abscissa 
of the cur4’e is then delenmned. This b the sampled value. The get 
of values obtained in this way is a random sample from the dbtri- 
bution. 

Tables of random variates from the normal dbtribution are avaU> 
able. SeeRA^^D (l9o5). Marshall reports jin Meyer, (1956, p. 10)1 
that at RAND the researchers “eurrently generate random variables 
h3^'ing the exponential distribution by taking the logarithm of random 
numbers." He observes: 

It is only gradually that there has devdoped a practical capacity to gen- 
erate random variables from a wide fli-sg of probability distributions. The 
choice of any particular scheme of course depends upon the economics of the 
computation sitnadon. 



Figuse IIJ. Converaon of a 
random BuwbcF to a random 
deviate from a specified dii- 
tribation. 
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Detailed discussion of this problem can be found in papers by Lytle 
and Butler [in Meyer (195S)J. 

Variance-reducing techniques 

Kahn [in Meyer {1956)] has provided an excellent survey of the 
principal techniques of reducing rariance of estimates derived from 
the use of Monte Carlo procedures. His presentation leaves little 
room for improvement, and hence this discussion is an Iconic mode! 
of his. The six most useful techniques of vari.anee reduction are; 

(1) Importance sampling. 

(2) Russbn roulette and splitting. 

(3) Use of rapcclcd values. 

(4) Systematic sampling. 

(5) Stmtified sampling. 

(G) Cortciiition and regression. 

The names of the last four consist of terms which )iavc appeared 
in earlier discussions. Their meaning in tins context is closely rC' 
lated to the meanings imputed earlier. Eacli of the six techniques is 
equally applicable to normal sampling problems, and several arc quite 
widely used in the normal sampling context. 

Kahn illustrates the first five techniques by an example which is 
very useful pedagogically but which, as he points out, is so simple as 
to be trivial, The problem is to estimate the probability of obtain* 
ing a “3” on a toss of two standard dies. By normal analytic pro- 
cedures we can determine this probability by obser%'ing that there are 
two ways of “making a ‘3’”: (2,1) and (1,2). Since there arc 36 
possible outcomes, the probability is or ^is- 

If tlie dies are tossed at random, or if we employ pairs of random 
digits to represent outcomes on each die, we would use a relative- 
frequency estimate of this probability, p, where 


n 



N is the number of trials, and n the number of occurrences of “3.” 
The variance of this estimate is given by 
, p(l - P) 

N 

The per cent error is given by 
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It U clear that, for fixed N, as p increases the per cent error decreases 
and that, for fixed p, as N Increases the per cent error decreases. 

(1) Importance Sampling. This is a way of artificially increasing 
p and adjusting for this increase in nmking the required estimate. 
Suppose, for example, that we use two columns of random digits. In 
addition to letting the pairs (1,2) and (2,1) represent a toss of "3,” 
we also Jet (7,8), (8,7), (8,9), (9,8), (9,0), (0,9), (7,0), (0,7), 
(8,0), and (0,8) represent a "3.” Wc now have 12 (or 6 times as 
many) ways of getting a “3.” The estimate of p will now be 

1 n 

The original per cent error would be 



The new per cent of error would be 



and hence is reduced by V^/V5 or by a factor of approximately 3. 

In general terms, sampling by importance can be stated as follows. 
The problem is to estimate the value of 

y=f »«/(«)*. 

where /(z) i.s a probability-denaty function (PDF), and z may be a 
vector. The problem is to find another PDF, hit), such that 

y -J's'WMo dz, 

where the estimate of Y is 

n 

and Zi, is a random sample from the population with PDF, 

b(2). 
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For the use of multistage sampling in the development of an impor« 
tanco sample sec Marshall (in Meyer (19oB, pp. 123 ff)]. For a dis- 
cussion of the use of the variance for measuring accuracy of the re- 
sults see Walsh (In Meyer (1036, pp. 141 C)). 

(2) BiMjinr; Rouletle and SpIiKmi;. Now suppose that we toss 
only one die at a time or draw only one number at » time. We know 
that only if wo obtain a **1” or a “2" on the first draw do we have 
any chance of getting a "3.” Therefore, no can fcparato those first 
draws on winch we do not get a "1” or a “2” (ipfit the sample) and 
make a second draw only for the first eet. This makes ft itnnccessary 
to make the second draw in ti»o thirds of the cases. Hence, the same 
variance can he ohtninctl with one third less total observations, or 
with the s-ame number of draws wc can decrease the variance. 

In some e.asea it may he neither ncccssaiy nor possible to ktU aft 
the '‘uninteresting” first draws, in which c-osc a random sample of m 
of the uninteresting cases can t>c scfectcd and tlio estimate based on 
these observations can Iw appropri.atcly weighted by w. In addition, 
each interesting case can be lrcatc<l as tliough it were n (n > 1) tuch 
cases, and the resulting c«titnatc can also be appropriately weighted 
by n. This procedure is called Aussuin roulette. The idea is duo to 
von Neumann and Ulam (19(5). 

Tlia i<ie.a of Russian RoulcUo aod Splitting is simitar to the sequential 
suopfisg sebemes of guih'ty control, tho^h quite di/Tercnt in deuil. It was 
first thought of in connection with particle diffusion problems. Particles 
which get into interesting regions are spht into n independent sub-partictes, 
each with one nth of the weicht of the original particle. Particles which 
get into uninteresting regions are, in effect, amalgamated into a fewer num- 
ber of heavier particles. In this way the e^cirfator aebtew his goal of allo- 
cating his effort seombly. [Kahn (1056, p. 147)] 

(3) Use of Expected Values. In cases where the sampling is done 
in two stages, even though wc cannot compute the relevant probabili- 
ties associated with each stage wc may be nblo to do so for the second 
stage. For example, wo may not know the probability of getting a 
“1” or a “2" on the first toss of a die but we may know this prob- 
ability for the second toss. Then wo need not use the second toss. If 
wc have liad a "1” or a "2” on the first loss ond know that tlie prob- 
nbibly of gcllii}g Ibe olber cm (be second loss is then we can esti- 
mate the probability of getting "3” on two tosses as 
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where n is the number of occurrences of "I” or ‘‘2’’ on the first toss, 
and is the total number of first tosses. This procedure is referred 
to as the use of erpected trolues. 

(4) Systematic SamplinQ. Now suppose that we reverse the situa- 
tion of the last example; that is, suppose that we know the relc^’ant 
probabilities on the first loss but not on the second. Wc would know 
then that, if we made 600 “first” tosses, the expected number of “1” ’b, 
“2’"8, "3”'8 “4” 'a, “5’"8, and "6”'8 would be 100 each. Therefore 
we can omit the first toss and assume for the first 100 of the “second” 
tosses that a "1" came up on the "first" toss. We would assume that 
a “2’' came up on the second 100 “first" tosses, and so on. This 
eliminates the need for the first toss and reduces the fluctuations in 
the proportions of "I" ’b, "2’”s, and so on, which would occur because 
of random tosses. 

The reduction of the estimating error by this procedure of sys- 
tematic sampling is generally small: between 5 and 30 per cent. But 
since it is easy to obtain this gain, it is foolish not to do so where 
possible. It does require, of course, prior knowledge of the sample 
size to be dranm. 

(5) Sfrofi/Icd Sampfinjj. The reflective reader will have asked a 
question about the preceding example: why make the last 400 tosses, 
since we know that if there is a “3” or higher on the first toss we 
cannot obtain a sura of “3” on two tosses? Therefore we ought to 
divide the total sample into two strata of 300 each. For the first we 
assume a “1" on the first toss, and for the second a “2.” 

In theory, this method could be as powerful as Importance Sampling. In 
actual practice, the fact that you have to sample systematically turns out 
to decrease sharply the number of pbces in which it c.aa be used. However, 
where it can be used, it is usually better than Importance Sampling and in 
any case never worse. Therefore whenewr the costs of the two techniques 
are comparable. Stratified Sampling is preferable to Importance Sampling. 
[Kahn (1956, p. 155)] 

(6) Correlation and Regression. To illustrate this procedure the 
same coin-tossing example cannot be used. Suppose that we want to 
determine the difference in average watting time for customers arriv- 
ing at a service point with two identical service facilities. It would 
be possible to conduct two separate simulations, one for each service 
point, but it is preferable to use the sampled data on arrivals and serv- 
ice for the first facility again in the second case. In the second case 
we would merely add a sample of service times of the second facility. 
Then we can compute the difference in waiting time for each customer 
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and obtain from ificse results an estimate of the mean difference in 
waiting times for tJie two facilities. 

This procedure can bo extended to cover any ntimber of scrnce fa- 
ciUtica. Tifc estifnaled me.m (lifTcrcoces would be plotted against the 
number of fneilitiea at the service point, and a regression line fitted. 

Not tm]y docs the procedure just «Je'C«bed reduce the number of 
samples which must Iw drawn, but «lso it reduces the kinds of chance 
fluctuations which can affect the results; that is, which can jacrease 
estimating error. 

Concerning the six tcchni(]ues just described Kahn (1950, p. 156) 
has the following methodological ob«crvation: 

Tcchnifiucs described under the licadiogs of Importance Sampling, Russian 
noiileiie and Splitting, and Stratified Sampluig have the property that in 
many calculations they will give a tremendous increase in efBciency if prop- 
erly used: if, however, the intuition of the calculator is faulty and he docs 
not use a reasonable design, tlien they can be very unreliable and actually 
inemro tbe vsriaoce. The other techsktues are more stable io that it is 
almost impossible for the experimenter to worsen the samplmg variaBce by 
misusing them, even if he baa a bad intuition. 

Variance-reducing techniques c.an in some instances yield startling 
reduetioos in the number of computations required to obtain a specie 
fietl level of precision. In a problem rci'orlcd by Arnold, Bucher, 
Trotter, and Tukey (in Meyer (1950, pp. SOff)), by tbe use of a 
proceihirc that combines splitting and what these authors call rondi- 
fi'onaZ rafcufdftons they arc able to obtain a gain of a factor of about 
hOeX) over standard simple random-sampling techniques. 

The following c.xtondcd quotation from Marshall (in Meyer (195C, 
p. 8) I provides a succinct summary of the methodological aspects of 
variance reduction in Monte Carlo procedures: 

Tho bemose m tlie speed of [computing] machines has tended to make 
variance reducing tccJuiiques relatively leas interesting, but has by no means 
thminated their usefulness. The effect of increased computing speed in the 
newer machines is to omke the cost of designing and ct^ing a problem in- 
crease relative to the cost ol maebine rusning time. Tbe use of variance 
roflucing techniques shortens running time but at the expense of (1) incrcas- 
iug the time spent in dedgning tbe computations so as to adapt the classical 
techniques to the particular picWem or in Iho invention of a new, more 
suiUible technique, and (2) complicatinE the coding because of the more ciab- 
ffnte hookker/xn^ and naJciflarinns ]JIk» trebniqurs usually require. On the 
whole, however, it there is one thing that would generally increase tie use- 
fulness of hfonte Carlo it b the di^ery of new varunce reducing tech- 
niques, or the application of knom* varianee reducing techniques as a matter 
of course to the ordinary nm of problems. Not only should these techniques 
be useil whenever it b economical to do so but, in addition, since the van- 
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once reducing tectnii^uec are not yet weU known there sbouJd be a bias to- 
ward using them even when they are not economical for the problem at hand. 
This is a way to learn about them for use in later and more suitable prob- 
lems. The use of new techniques m marginal cases is almost always justified 
as a method of building inteUectuaJ cajntal. la the Jong run one woid sup- 
pose that real thought on the design of Monte Carlo problems will be con- 
fined to problems of a basically new type whenever they first appear; stand- 
ard variance reducing techniques will be avadable, and used, for other prob- 
lems on the basis that sub-routines for computing common Imctioss now 

Here in Marshall's statement concerning "building intellectual capi- 
tal" we liave e.xpljcit recognition of the obligation of scientists work- 
ing on either pure or applied problems to take responsibility for the 
methodological and technical development of research procedures. 

By now it is probably clear to the reader that the problem of re- 
ducing variance in Monte Carlo simulations is very similar to that 
faced in any application of sampling procedures: 

For ttus reason, it is very valuable to have professional statisrical help is 
desigmng these calculitloDS. However, if one has to choose between a person 
who is mainly interested in statistics and one who is mainly interested in the 
problem itself, experience has shown that, in this field at least, the Utter is 
preferable. Tbis last remark U not intended as a slur on statisticians, but 
dmply to amplify a comment made eariier, that, "the greatest gains in vari- 
ance reduction are often made by exploiting specific details of the problem, 
rather than by routine application of general principles." [Kahn (1950, 
p. 190)] 

Variance and Computers In Monte Carlo Simulation 

In some instances in which a statistic of two or more distributions 
is taken jointly, the variance of this statistic can be determined ana- 
lytically. In most instances in which simulation is required it can- 
not be. As a consequence the researcher cannot usually determine in 
advance how many simulation “runs" he will need. For this reason 
the application of multiple sampling (at least double sampling) is 
particularly well suited for simulation. ^Vhen the simulation is done 
by hand, the variance of the estimate sought can be recomputed after 
each trial. When a computer is used, however, it is costly and time 
consuming to divide the simulation into stages. IVhat is frequently 
oi’erioc^ed, od the ether band, ia that it Bsay be mare assrtij' and time 
consuming to select arbitrarily a sample size and make too many or 
too few observations. Even where a computer is used, several hand 
runs of the simulation are usually required in order to check out the 
computer program. These runs can also be employed to arrive at 
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at least a crude estimate oJ the variance, which can then be used 
to estimate the required sample site. Given an estimate of the vari- 
ance, the procedure described in Chapter 7 for determining optimal 
sample sise can bo applied, perhaps with modification to suit the 
characteristics of the situation. To determine what estimation method 
to choose, the procedure described in Chapter 8 can be employed. 

The question as to n'hetiicr or not a computer should be used in a 
simulation involves consideraUoos of economy as well as of accuracy. 
In corople-v simulations the opportunities for nonsampling errors are 
considerable. These errors iiave the aame characteristics, in general, 
as obser\’er errors and can be analysed in the same way. {See p. 
206.) The cost of using a computer involves not only running time 
but also programming and the inevitable “debugging.” The simula- 
tion must be more complex and lengUiy than we normally suppose 
before the use of a computer involves less human labor than does 
hand computation. 

The Los Vegas Technique: Nfargbal-Cost Analysis of Simulation 

SimuiatioQ, as noted earlier, can be user) to evaluate terms in a 
model so that it may be solved or it may bo utilised to find an ap- 
prosimate solution. Where there are a /laite number of possible 
solutions, a solution can always be found in principle by exhaustive 
trial and error. But for most problems of interest, even with the 
help of large computers such brute-force tactics ore seldom practical. 
It is obviously preferable to try to find a procedure which will tend 
to yield closer and closer approximations to a solution. In situations 
in which the model must be solved at different times for different 
values of the uncontrolled variables, a converging simulation process 
(sometimes called the La$ Vegas technique) may be designed as 
follows: 

(1) A procedure for simulating llie model is developed, one which 
yields a measure of output for a specified set of values of the con- 
trol variables. 

(2) A procedure is de\’elopcd for examining the performance of 
t!ie simulated system to locate trouble spots for correction. 

(3) Rules are developed for changing the values of the control 
variables in a way which (on iotiutivc or other grounds) appears to 
yield improvements. 

(4) If the system is stochastic, repeated trials for a selection of 
sets of values of the confrol variables arc run to provide an estimate 
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of within-nm variance. Using the estimate thus provided, a determina- 
tion is made of the number of mns (Jb) required per set of values to 
be able to detect differences of a specified magnitude with a specified 
probability. 

(5) Starting with an initial act of values selected on the best basis 
available, k runs are naade. The results are examined (step 2 above), 
and the decision rules are applied (step 3). A new series of k runs 
is made for the new set of values; and so on. 

(6) The average outcome of each scries is plotted against the 
number of the series. (See Figure H.2.) 

(7) After a large number of series, starting with the first, each 
point which represents a belter performance than any yielded up to 
that point is circled. 

(8) A line is fitted to the circled points. This maj' be done by 
free hand. 

(9) The cost of k runs for a set of values of the control variables 
is determined. 

(10) The first point on the curve at which the expected increase in 
performance for the next scries is less valuable than the cost of the 
run is determined. 

This, then, establishes the number of series which should be run. 
The better the scaKh procedure and decision rules (steps 2 and 3), the 
fewer series of runs will be requited to attain some specified amount 
of improvement over the first series. The length of the simulation 
also depends on how close to optimal is the first set of values of the 
control variables which is chosen. Here it may be possible to use 
even a crude analytic approximation to considerable advantage. 

Since there is likely to be considerable variance in the number of 
series required before the break-even point is reached, it is better to 



PiBCBB 11.2. A Las V^as 
plot. 
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use the Las Vegas technique sequentially than to use one set of scries 
to determine how many series to run on another. 


GAMING 

A simulation in which decision making is performed by one or more 
real decision makers is called gaminQ. This terra is sometimes re- 
stricted to simulations in which two or more competing decision 
makers are involved. The term, however, will not be so restricted 
in its application here. 

Gaming has come into increasing use in Uic last decade, particularly 
in the study of complex military' and industrial operations. It is now 
beginning to be used in the study of governmental problems at the 
municipal, national, and international levels.* Gaming, particularly 
in the military contest, has a very long history’, which is described in 
detail by Young (1957) and Thomas (1901), hut its use as a research 
tool U quite recent As Iloggatt (1959) olisen'cd, its principal appli- 
cation has been ns ‘hi tcachiog device and In making the play of the 
game interesting for Uic participants" (p. 192). It is also being used 
increoaiogly to select as well ns train {lersonncl, to familiarite per- 
sonnel with the operations of a compIc.t tystem, and to demonstrate 
A new idea about a complex system. For discussion of gaming as 
oUicr tlian n research tool see TI>om.ss and Drcuier (1957) and Cohen 
and Ahenman (1050). Bibliographies on gaming may bo found in 
Malcolm (1900) and Shuhtk (I960). 

Tlie uses of gaming in problem-solving research fall into three gen- 
eral classes; (1) to help develop a decision model, (2) to help Cml the 
solution to such a model, nod (3) to help evaluate proposed solutions 
to problems modeled by the game. Gaming can aid in constructing a 
model by proi'iding a basis for testing tlie reJevanco of variables or 
the functional form of the model (ix., the relationship between Ifie 
variables). It can also be used both to help uncover possible coiir-w 
of action ami decision strat^ie*, and to eomp.irc the alternatives, 
In cases where n completely specified course of action or decidon 
procedure cannot he derived analytically from a rawief. but a 
tially specified action or procedure can, the effect of the action or 
procedure may' be dctcrnilned by gartting. 

•Use of gaming in municipal prohlcin* is lacing bind'’ by Prof. Nafluui Cnio- 
itcin kt the Unnersity of Pitl-lmriA M»it Prof. Richard .>frfrr at the fni«cr»i(,v 
of Miehigan. Aft canmplo of »U ■pplicatkm in the *ti«!y of intcm-iiioiwl i>ni>' 
lews Can be foimd ia OueUiew (1939), 
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The application of gaming in pure research parallels that in ap- 
plied research. It can be used to help develop theory, to suggest 
hypotheses to be tested, and to test hypotheses suggested by the 
game, other experience, or theory. It should be noted that the use 
of gaining in pure research involves decision making, but not in the 
conte.Tfc of a particular problem or set of problems of current practi- 
cal interest. 

Gaming is essentially experimentation in which the behavior of de- 
cision makers is observed under controlled conditions. It differs from 
most psychological and social experimentation only in that the con- 
ditions under which the “play” is observed represent some situation 
outside the laboratory about which knowledge is sought. The experi- 
mental situation, then, is deliberately constructed as an iconic or ana- 
logue model of a type of situation of interest. 

The principal methodological problem in the use of gaming lies in 
the process of inferring from the experimental to the “real” situation. 
We will consider this problem after we have illustrated and discussed 
the various research appheatioos of gaming. To facilitate this dis- 
cussion a particular game (developed by the author and several of 
his colleagues) will be used for illustrative purposes. 

The game to be described was developed as a pure-research tool. 
It is intended for testing the eSeet of certain types of variables de- 
scribing o^aoizatlons on their performance. Tlie results of the ini- 
tial use of this game are described in detail in Clark and Ackof! 
(1959). The game is currently being employed to help develop a 
mathematical theory which explains the performance of an organisa- 
tion in terms of the variables which are being investigated simultane- 
ously. This use of gaming is by no means unique; for example, see 
Hoggatt (1959). 

An Example of Gaming 

There are three basic operations to be performed in this game: 

(1) Purchasing: buying letters of the alphabet printed on small 
wooden “tiles” (which are similar to those used in the game of 
Scrabble) . 

(2) JIanufacturiag: forming words out of these letters. 

{3} SeJJiog; playing the vords on a board. 

Each of these operations is the responsibility of one player, and hence 
the mimmal team consists of three persons. 

Each operation involves a cost, and selliDg yield.s an income. The 
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objectivo of the group is to maximitc its profit over a series of plays. 

In addition (o the Purchaser, Mooufseturer, sod Saiesoisn, there 
rony bo an Executive. Tlte positions of the players around a U> 
shaped fable arc shown in Figure J1.3. The Purchaser, Manufac- 
turer, and Salesman are separnted by a visual barrier (made of 
Masonite or plywood) through Iho bottom of which is a slot. The 
lettered tiles can be passed Uirough this slot by sliding them along a 
rack that is provided. The Executive, if present, is the only player 
who can sec all that is going on. At the beginning of a trial, or 
run, tlie team is supplied with “550" in chips. As the Purchaser 
obtains letters and as the Manufacturer forms words, they must 
pay the experimenter for these operations out of the team’s supply 
of chips. As the Salesman plays these words, the team receives 
income in chips, Jess the S.ilcsnian’s expenses. 

If the team has to "borrow money” at some point in ita operations 
it can do so, but it is charged |l for every 52 that It borrows. 

The Purchaser has before him a rock on which consonants are 
grouped, valued according to their frequencies of use in the English 
language, as follows: 

52 54 56 5S 

JKQXZ PGPVW CDPLM HNRST 

In one version of the game, consonants are obtained by buying 
adjacent pairs within a group.* This is the only way in which they 

* In other versions of the eauc “quantity disconnta" are provided for large 
purchases. 
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con be obtained. The cost per letter is shown above each group, bo 
that the pair HN, for example, cost $16, wljereas the pair QX costs 
only $4. 

As the Purchaser buys pairs of consonants, he pays the experi- 
menter for them, records the transaction on bis expense form, and 
then forwards them to the Manufacturer, placing them on the rack 
which slides through the slot in the barrier. 

The Manufacturer has before him the vonels, and the letter 
As he receives consonants, he may combine them with the vowels in 
any fashion to make words. When he forms a word, he pays the 
experimenter $S per vowel used, and records tills operation on his 
expense form. At this point, the experimenter can declare the word 
unacceptable if it is an abbreviation, a proper noun, or a misspelling, 
and remove it from play. 

The blanufacturer cannot accumulate more than six consonants. 
If more than sue arc available (have already been purchased), he 
cannot use those most recently bought until the six first acquired 
have been employed in words. He may “scrap” any of the six avail- 
able consonants and obtain another from the Purchaser. A scrapped 
tetter cannot be used subsequently. 

As words arc formed, they are forwarded to the Salesman. He has 
before him a board (sec Figure 11.4). As he receives words from the 
Manufacturer, he plays them (thou^i not necessarily in the order in 
which he receives them) on the board, according to the following 
rules: 

(1) Play must begin in the upper left-hand comer of the second 
row of the board. The first row is filled with a random sequence of 
letters. 

(2) The sequence of play is across the second row from left to 
right, then across the third row, and so on. 

(3) At least one space must bo skipped between words. 

(4) Every word played must have at least one letter in it that 
matches a letter directly above it (in either tlie row above or any 
previous row if the spaces between the matching letters are not oc- 
cupied. 

For each word that the Salesman plays, the team receives an in- 
come of $10 per letter, less $6 per space skipped. 

The Executive is responable for coordinating team effort and di- 
recting strategy. There are no restrictions placed on his activity 
He may function as a trouble shooter, helping out where bottlenecks 
occur in the course of play, or as a communications center. His 
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orders must be carrietl out by the other players. If a player refuses 
to follow nn order, the Executive may "fire” him (for the remainder 
of that run) and fill in at the poi«ition himself. 

Flay continues in this manner for a period of 10 minutes. At the 
end of this time, expenses and income are totaltn) from the records 
kept by the players, letters are retuiued to the supply racks, and a 
new set of letters is placed randomly at the top of the board. 

An 8-minute period between rounds allows the players a discussion 
period of about S minutes, in addition to preparation for the next nm. 

Each player, except the Executive, is scored on his individual per- 
formance. Ilis reward (in money) depwids on the group’s perform- 
ance and his individual performance. The Executive receives a speci- 
fied portion of tlie group’s earnings. Tlic Purchaser’s objective is to 
minimize the average cost per letter bought. His score Is based on 
the extent Co which he meets Ibh abjeeiiiV: The A/snu/settfrer’s el^- 
jective is to minimize the coat per word, and hence to minimize use 
of vowels. The Salesman’s objectn-e is to minimise the cost of sal^, 
and hence the number of skipped spaces. These objectives create a 
conflict of interest. 
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If the Purchaser seeks to minimize his costs by buying inexpensive 
consonants, the Manufacturer will usually have a harder time making 
words and will have to use more vowels in the process. Also, the in- 
terest of the Salesman in matching letters without skipping more than 
a minimum number of spaces will often be in conflict with both the 
Purchaser’s and the Manufacturer’s attempts to keep their costs low, 
since he may call for words using more vow els or expensive conso- 
nants. Thus some compromise among the player’s individual ob- 
jectives is necessary for effective group performance, and one of the 
most general strategic problems for an Executive is to determine 
what sorts of compromise will be both acceptable to the players 
and efficient for group performance. 

Numerous aspects of this organizational .situation are subject to 
experimental control. Some of the more important are the fol- 
lowing: 

(1) The presence or absence of on Executive, 

(2) The fact that the players may have fixed roles or may be 
rotated. 

(3) The fact that the players may be trained ns a team or in func- 
tional groups. 

(4) The amount and type of communication permitted. 

(5) The method of compensation. 

A brief description of one of the experiments and its results is 
given in Clark and Ackoll (1959, pp. 291-293) ; 

... six 4-tnaii teams were formed. . . . 

Two teams were designated “cooperative" teams. The members of these 
teams were told that they would be competing against other teams; that the 
team with the highest score would receive a 330 prize, to be divided evenly 
among the members of that team. Two other teams were called ‘'scmico- 
operative” teams. (These designations were used only among the experi- 
menters.) Members of these learns were told that they would be awarded 
prize money on the basis of bow well they scored ia their position as com- 
pared to the scores made by players in corresponding positions on other 
teams. That is, the purcb^r was really competing against other pur- 
chasers; the manufacturer against other manufacturers, and so on. 

The third pair of teams were called “conflict" teams. Members of each 
of these teams were told that they were competing against each other for prize 
money. That is, if the salesman did better at his position than the other 
players at theirs, he would win all the prize money for that team. The other 
pfayers woufrf wia nothing. 

Each team went through three sesaons. The first consisted of training 
and practice runs. During the last two, each team engaged ia 10 runs. 

A f-test showed that the two “cooperaUve" teams bad mean team scores 
that did not differ significantly. The means of the two “conflict" teams were 



DEItmXO SOLVTIOKS mOM MODELS 369 

almost significantly dilTercnt at the Oitt Icvd, indicating that it would be 
preferable not to combine these two groups in sutewjurat analysis. For the 
lack of other teams in this classifiatitaj, however, the scores for these two 
teams wore combined. 

Two main analysis-of-variancc computations were carried out. One in- 
volved two "levels” of competitive effect (cooperative and conflict te.ama); 
the second, all three. The "column” contribution to the total variance, tlien, 
represented the effect of varyit^ the competitive make-up of the team. The 
"row" effect represented the <^ect of trial number, i.e, a ‘fleaming" (or, 
more precisely, a "team improvement”) effect. In both analyses, a column or 
"typc-of*tcam" effect waa between the 005 and 001 significance levels. With 
all sit tcamit, the learning effect waa significant at the 0 05 level. Wuh just co- 
operative and conflict leaim*. the learning effect waa not significant. This 
rcllccts the general lack of improvement in team scores apparent for the two 
conflict teams. Where the results for all ax teams are included in the anal- 
ysis, the team improt'ement effect for the cooperative and scmicooperativc 
team overbafances its tack m the conflict teams. 

A distribution free test, the "Wileoron Matched Pairs Signed Rank Test,” 
resulted in finding a difTcrence between "cooperative" and "conflict" team 
ecorea significant at approrimately the 0X>5 le%'el. 

The tame noalyris was carried out for "number of letters played". . . . 
The "tiTie of team" effect was found significant at the ODl level. This oc- 
eurreace, in view of the set of rule* for tte game . . . euggests that co- 
operative team strategy has been to emphasise word output at some expense 
to the Rtinimimtioo of costs. Where more conflict among group members 
occurs, and hence more attention may be given to individual costs, total 
WQr<i output may fill conriderably, but net team income may not fall pro- 
portionately. In fact, a calculation of the avenge profit per letter played 
finds one of the "conflict" teams with the highest score. 

Since the game under discussion was developed as an experimental 
tool, it was designed to be flexible enough to permit study of ft wide 
range of variables. TJjia is best seen by noting that the team which 
has been described is an organuationaj module which can be com- 
bined with similar modules in many ways to create situations for 
testing a wide range of hypotheses. To illustrate these possibilities 
consider the following two situations: 

(1) To obser\-e two organlrstions (s.ay, of a business type) under 
conditions of inlaise competiUon the teams are arranged as shown 
in Figure 11.5. Here the Purclinsers must bid (in open auction or 
closed bidding) for raw materials (letters), and the Salesmen must bid 
for the right to make the next sale. In this situation teams can be 
“pat oat of business.” Behavhr ia eitaatios is wflceaWy dif- 
ferent from that shown when a team plays without competition, 

(2) Two common principles of njanagement are (o) management 
by function (horizontal structure) and (b) management by end prod- 
uct (vertical structure). To test the relative advantages of these aJ- 





Figure 116 . Setup of game for teatine laoruoatal and vertical etnicture of an 
organization- 
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ternatives U)e following siluaHon can be crealcfl. Let the organiia- 
tional modules be physically arranged aa shown in Figure 11.6. Ver- 
tical structure is obtained by ha-nng Manager 1 control Team 1; 
Manager 2, Team 2; and Manager 3, Team 3. Horizontal structure 
is obtained by having Manager 1 control the three Pureliasers; Man- 
ager 2, the three Manufacturers; and Manager 3, the three Salesmen. 
In addition, a super or corporate executive can be established to con- 
trol tliese three managers. 

Tliere is no limit to the complexitj' of organization one c-an develop 
by manipulating tliese mothilM. 


The Limitations of Gaming 

In many instances games have been developed without any clear 
notion as to how they can or should be used, nationalizations for 
tlie effort are occasionally offered, but in general these games have 
been developed either for the scientific exercise involved or for the 
purpose of exploring the uses to which they can be put. Claims have 
been made for games as u.teful training and persosael-sclectioa de- 
vices and for demonstrating the operation of complex systems. As 
yet, however, there has been little or no controlled evaluation of their 
use for any of these purposes.* 

Thomas and Deenicr ^957) have provided an extensive discussion 
of tlie methodological problems associated with the use of gaming os 
an applied-research technique. The following quotation is from their 
article (pp. 19-21): 

Beyond [the] djfficuliy of kaowlng when one has solved the "riglil” prob- 
lem, there is the difficulty ... of knowins when one has solved any problem. 
... In gaming, genmWy, there is no way of knowing with certainty when 
a sample of plays is both ElratcgicaDy and statistically adequate for a re- 
quired decision. . . . 

Despite the absence of logical proof, operational gaming inspires its practi- 
tioners with a remarkable fonfirfence in its results. Sometimes an impIauaWc 
result is accepted with special rcltsb because of ita implausibiliiy. Tlie plaus- 
ible results are often accepted as bring now beyond dirpute. . . . 

In the formulation of a game one is beset by conflicting objectives that 
induce an ambivalent attitude toward elahonition. On the one Land, it is 
dearable to formulate n game the Bolutkm of which u highly relevant to tlie 
eompelilive situation Iwing investigated. Tljis consideration encourages one 
to rtdd more and more details in an efftot to acquiir rtalbin. On tlie otlier 
hand it is desirable to formulate a gsme the solution of which is posable, 

•Sec Thotnua and Dccmer (1957> for adetriled diseiswion of this point. Alw 
eee Cohen and Bhcntnaa (!SOO>. 
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at least to the rcqtJircil accuracy TIh« constraint icndi to inliiMl llic aiWi- 
tion of details. ... 

Tlie conunon tendency toward exccafrire elaboration in opcnitionil g-uninjc 
may be regarded as another instance of that misplaced cmpliasa on the “ap- 
pearance of reality” ... it is an easy mistake to make. Tlie eearch for lbs 
‘‘essence of reality” is anluous and difhcult, the goal difficult to recognize. 
One often feels that by ineor;x>ratiDg a few more details in a model of re.ility 
be is that much more certain of cajilunng the essence. When, as in opera- 
tional gaming, tie increa.'^ed thffinilly of colution easily escapes notice, the 
temptation to enlarge the model liecomes all the greater. 

But this temptation to elaliorate should l>e die more strongly resisted in 
gaming. For to yield is lo court ddusioo. Not only is there the doubly di- 
minished effectiveness of coluUon mentioned before as a consequence of ex- 
cessive elaboration, but there u also anotbef difficulty that arises in inter- 
preting the results of garrung. One tends to forget that (he game is not 
reality itself. ‘The "appearance of reality, ” ro a«cfuJ in te.sehlng, becomes 
dangerous in application. 


Gaming: Analogy or Analogue? 

Tlie fundamental weakness In current gaming lica in the inability 
to draw strong inferences from the piny of the game to decisioos in 
tlio situation which the game models. Tliomas and Dcemer argue 
that the inferences which arc drawn arc weakened bv the complexity 
of tlie game. A model, whether on cqu.ation or “a game, is always a 
simplideation of reality, and for this reason only is it useful in science. 
It is important, how'ever, to understand the nature and significance 
of the simplification because only then can we justify inferring from 
the model to reality. 

In current applications of operational games there is, I believe, a 
tendency to confuse the use of analogies and analogues. These are not 
the same thing; we can usually draw only very weak inferences (if 
any) by analogy. The inferenees being drawn by analogy, however, 
are sometimes given a degree of credibility they would dcserx'e only if 
tliey had been derived from manipulation of an adequate analogue or 
other type of model. 

It is important to understand the diltcrenee between an analogue 
and an analogy as these terms are used here. In both an analog>' 
and an analogue w’e use one situation as a model of another. The dif- 
ference lies in what we know of the correspondence of tlie models to 
the “real” situation. In an analogy wc know only that two situations 
have certain properties in common; we know nothing about the cor- 
respondence of the structure of the two situations. That is, in an 
analogy we do not know the function /, which relates the outcome to 
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the variables, and lienee do not know Iww well or badly U corresponds 
to the structure of the real situaUon. In an naalosue we self-con- 
sciously design into the model a structure tliat, based on analysis or 
cxperinienlatioD, we believe to correspond to some acceptable degree 
with the real one. 

The strucluro of a game corresponda to the structure of the modeled 
situation to the extent Uiat the same types of decisions yield the 
same performance in bolJi situations. SucJi correspondence of output 
for related input must be establislictl before inferences can be drawn 
from the game to the real situation. 

^\'e simply cannot atgue from a correspondence of properties to a 
correspondence of structure. Wc can construct a game and manipulate 
its structure until the relationship between its inputs and outputs 
corresponds to that of Uie real situation. In this way the game is 
used to explore structumi relationships nhicb yield a particular kind 
of input-output relationship. But since a particular input-output 
relationship over a certain set of values of Inputs may be produced by 
H large number of difforcot structures, it is dangerous to infer from the 
game input-output relationships involving inputs of values dilTerent 
from those that have been tested. Therefore, it is also important to 
establish the range of inputs over which the structure is asserted to 
yield outputs ccYfcsuonding to the real situation. This last observa- 
tion is similar to one that a linear approxim-ation to an S-sImped curve 
over a certain region of values of the independent variable may be 
good, but that it generally fit* badly outside that range. 

The more aspects of reality which are represented in a g-ome, the 
more difficult it becomes to analyre its structure (i.e., to represent it 
by a tnathcmalical model). On the other hand, unless enough of the 
relevant aspects of reality are included it cnimot be an adequate model 
of reality. Gradation between e.\ccs3ive simplicity and complcxi^ 
can be attained only by experimenting with the game itself. Con- 
sldemhly more time is generally required to develop a game that will 
bo useful in problem solving, than is required to use the game once 
it is developed. 

It a game fails to correspond adequately with reality or if the 
degree of the correspondence is cliineult to estsblislr, the rese-archer is 
likely to become interested in the game for its own sake and not as 
an analogue. At this point the gaming either becomes "ordinary” ex- 
perimentation, or it is used for othei-tban-research purposes, such as 
training of personnel or demonstraUoD of ideas. 
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A Game Used Inferentially 

Some games have been constructed winch can be used to infer ap- 
proximate solutions to real problems. These are generally simple 
games which involve a mathematical model of a major portion of 
the "real” situation. Such games are less dramatic and more modest 
than those which receive most attention in the literature, but they do 
point the way to the productive use of gaming for more than scientific 
exploration. The following is a simple example of such a game devel- 
oped to help solve a noncompetitive problem. 

In an industrial problem reported by Hare and Hugh (1955) it was 
necessary to find the order in which items requiring production should 
be processed over an assembly line. The setup costs * associated with 
each product depended on which item preceded it over the assembly 
line. The problem, which was to minimize the sum of the setup costs 
subject to certain inventory requirements, could be represented by a 
matrix in which the cost of making each product after each other 
product was shown. The matrix was not symmetrical, since the cost 
of setting up for product A after product B was not necessarily the 
same as for setting up for product B after product A. It was recog- 
nized that this was an "asyroroelricat traveling-salesman problem-" 
A general solution to this type of problem is not yet available. Study 
of the problem revealed several decision rules which appeared to yield 
lower costs than one would expect by using intuition and experience 
to sequence the production runs. The rules, however, did not com- 
pletely specify the decision to be made in any situation. Some judg- 
ment by the decision maker was still required. 

The researchers replanned the production of the last three years, 
using the proposed decision rules and their judgment where required, 
and compared the resulting costs with those actually incurred. A 
substantial reduction was obtained. The question remained, however, 
as to whether sucli improvements could be obtained by the people 
who actually planned proiiuetion in the plant. A game was set up 
involving the rescheduling of production over a three-year period. 
The people who actually had scheduled production over that period 
were taught the new decision rules and asked to reschedule production 
over the period. They did so, and the results showed the same im- 
provement obtained by the researchers. On the strength of these re- 
sults the rules were adopted and subsequently showed a continuing 
improvement over previous methods. 

•Costs incurred in aeijusfing ihe machmety for a production run. 
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Few would argue with tlic inference that the improvement in per- 
fonnance obtained by use of the decision rules in tins game was a 
legitimate basis for forecasting an improvement if the rules were ai>- 
plied in reality. The confidence one has in such an inference derives 
from the adequacy of the motfel of the problem silnation: tho cor- 
respondence of the game's structure to tJie situation it modeled. 


Conclusions on Gaming 

Gaming is simulation which involves “Jive” decision makers. It 
has boon used jirimarily where large complex systems are involved, 
systems whose structures arc not thorougldy understood. In such 
situations tiie principal justifiable use of gaming jj tlie cjqiloratian 
of structural relationdups. Results obtaincil from the game should 
be treated as suggestions or hypotheses which should be more rigor- 
ously tested. 

Jn ffoioe eitvithtts H msj' bo pos^iWe to eomtfuct an anafyt/c 
(analogue or sj-mbolic) model which fits a major part of (he "reality ” 
Combining such a model with a game can yield approximation* to 
solutions of problems which can be accepted with confidence, Such 
use of gaming, in eflcct, is controlled experimentation. 

Gaming should not be considered os a substitute for analytic 
model construction, To Uie contrary, H should be viewed as a wnj' of 
obtaining information which can be used to gcoeratc models where 
analysis of or experimentation on the ‘'real'' situation is impractical 
or impossible. 


SUMMARY 

In this chapter «c Imvc been primarily concemcil with various 
ways of deriving solutions from models. First we considered analytic 
(ceimiquea wdlh which an abstract symbolic expression for optmi'izing 
values of controllcil variables can bo derived deductively from an ab- 
stract s>Tnbo!ic decision model. By appropriate Buh«titution of num- 
bers for symbols in Ibcsc expressions the numerical optimiring values 
of the controlled variables can be obtained. 

In Forne c-ascs the rBathcmatrcal form of the decision model pre- 
cludes Fueb an abstract deduction of a solution to the problem. A 
solution may be obtained, however, by substituting (he appropriate 
numbers for the parameters in the decision model and tlicn "solving” 
numerically. In some such cases it may not be jws-siblc to find 
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directly the best values of Uie controlled variables, and n number of 
trials may be required to identify them. Such trials and errors may, 
in some instances, be conducted so as to converge monotonically or 
trendwise on the solution; if so, they arc called iterations. 

In still other situations the outcomes that result from using epccific 
numerical values of tlie controlled variables cannot be calculated, 
usually because the measure of performance is a parameter of a sta- 
tistical distribution of outcomes xchieh arc in turn a function of taro 
or more probabilistic processes. In the.®c eases we may be able to 
"run the S}’8tcm on paper” or simulate it in some other way so as 
to generate a set of outcomes under specified conditions and obtain 
the required estimates from these obscra'ations. Such simulation is 
experimentation on a representation of the phenomenon under study; 
that is, vicarious experimentation. 

One important advantage of simulation Is that it allows us to study 
the transition of the phenomenon from one slate to another. Such 
transitions can sometimes be studied analytically, but it is usually 
verj’ difficult to do so. 

Finally we considered problem situations which contain decision 
makers whose behavior cannot be effeclivcly represented in decision 
models. In these cases simulation can be conducted in which decision 
makers participate directly. This type of simulation, gaming, was 
summarized in the preceding section. 
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EXPERIMENTAL 

OPTIMIZATION 


INTnODUCnON 

Tlicrc ftr« two situations in which experimentation may be used 
effectively to locate the optimum of n decision function. Fir«t. there 
is the situation in whicli simulation (vicarious experimentation) is 
necessary to find a solution to a model. Second, there arc problem 
situations for whic!> an adequate decision model cannot be con* 
structed. This may be due either to our Inability to identify the tin* 
contxoll.abjc vari.sbles (T/) or to n J.ack of knowledge of the under- 
lying function (/J which rclalca ll>e outcome to the controllable and 
uncontrollable variables. It may still be possible to find an optimal 
solution by means of experimentation. One cotild, of course, resort to 
trial and error, but such a haphasanl procedure is usually too costly 
and time consuming. Fortunately, by controlled triafs—that is, by- 
experimentation — the researcher jnay obtain good pstimatea o! the 
optimal values of control variables in a lime- and c(rort*con«erving 
way. 

Designs for seeking optima experimentally fall into two classes: 
simultaneous and sequential. In simultaneous designs all the combi- 
nations of variable values at which observations arc to he made arc 
selected in advance, wlicreas ia scqucotial designs only a few poinb* 
arc *0 selected. Selection of the remaining points is m-ide after some 
observations and i^ Ij.a.'od on the infonnation obtaineii from these 
379 
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earlier observations. Combinations of llicse types of design arc also 
possible. 

Although we will consider the relative advantages and disadvantages 
of these types of design in later discussion, some general observations 
arc in order here. 

In simultaneous experimental designs the observations may be made 
either simultaneously or sequentially. In sequential designs most, if 
not all, of the observations ate made in sequence. Where observations 
are made sequentially, the same “subject’' may be used over and over 
again or different “equivalent subjects” may be employed. For ex- 
ample, in experiments to determine wbat combination of altitude and 
speed of a bomber yields minimum dispersion of bombs dropped on a 
point target, different bombs and even different targets may be u^ed. 

Where the observations arc made simultaneously, different "equiv- 
alent subjects” may be usctl. Either of these requirements or both 
may be impossible in some problem situations. In marketing research, 
for example, it may not be possible to duplicate a specific market. 
Furthermore, a particular market place may be changed by an experi- 
ment 80 that it cannot be used twice. If a sequence of observations is 
to be made on a single subject, then we have the general requirement 
that the subject return to the same initial stale after each e.xperiment. 
This means that such changes in outcome as follow the changes in the 
controlled variables must not be affected by previous values of the 
controlled variables; that is, the effects roust not be cumulative. For 
example, current sales of a consumer product may be the effect of past 
as well as current advertising. It may be necessary to have long 
periods of no change in the controlled variables so as to allow the ef- 
fects of previous states to diminish or disappear. This is contrary to 
the practical experimental requirement that changes in the controlled 
variables can be made rclalively frequently. 

In general, subjects which can be brought into a laboratory lend 
themselves better to eillicr simultaneous or sequential experimentation 
than do subjects which must be observed in their natural habitat. 

In either type of design (simultaneous or sequential) the uncon- 
trolled variables must either remain constant or vary in a statistically 
stable way. That is, the variations in observed outcome must be at- 
tributable either to the variables subject to experimental control or 
ta tlie ffuctuations (experimental errocel due tn the uneowtiolled vari- 
ables. These errors must be random and measurable. Even if this 
constancy or stability is attained during the experimentation, it may 
not continue in the “real” situation. The optimum obtained in experi- 
mental optimization is applicable only as long as the environment of 
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tlie phcnoinona umicr etudy remains constant or stable. Therefore, 
experiinentnl optimuation at best is not a complete substitute for 
optimization based on a model. The latter can always be appropri- 
ately adjusted when the uncontrolled variables change, but this can- 
not be done with optima derived Stom eipcnmenta alone. 

Simultaneous cxi)crimcnta can usually be conducted over a eborler 
time span than sequential expenmcnla and hence are leas subject to 
distortion due to instaidlity time trendsj of the uncontrolled 
variables. But the results of both types are equally subject to such 
instability when applied to the real situation. 

Finally, it should be oliservo! that, in sequential experiments on 
a single subject, no mu>t be able to change the values of the con- 
trolled variables at will and, in general, more frcrjuently than tjify 
would normally be changed. 

We shall consider two types of simultaneous design: random and 
jaclorial; and tn'o types of sequential design: the tingh-fitcior and 
the steeptsl ascent. Only for the loclhod of steepest ascent has n 
relatively extensive literature l»een dcvelopc«I. The otJier designs ap- 
pear in the literature more os suggwtioni tlrnn as well-devciojwl 
techniques of optimization. This is due in p.art to tlic fact that the 
motliod of steepest ascent is conridered to be tJic best available, and 
hence, since its rm>cnl«iion in 19'»l. investigation of the other ap- 
proaches has been curtailed 

In the dispussion of liiesc four techniques we will be coccemnl 
primarily with tlidr iimlcrlying logic rather than with pn>cedur.al 
details. Before turning to the terhniquea, let us set the stage a bit. 

Consider again the idcalizotf giwral form of a dectston model: 

l<i<m, l<i<n. 

We observe that the function can be plotted in a space of m + r* -f I 
dimensions. If the I'j (uncontrolled variables) arc not known, then 
we would have an incomplete nwlel of the form: 

r-m'd- 

This ran be plotted in a space of in -b 1 dimensions. In experimental 
optimization we deal with such inronipletc models and assume that 
the )*, arc constant or statistically eubic, Si)vfifjcal!y, the "residual" 
or "chnnee” effects due to the uncontrolled variables arc assumed to 
be normally distributed, to l>c independent, nnd to have tlie same vari- 
ance for all observations. Further, it U assume*! that Uie mean of the 
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residual efTccls is rero; tliat is, tn the long nin the negative and fwsi- 
live efiects tend to cancel each other. 

For illustrative purposes wc shall consider only cases involving 
tn'O control I’arjables, A'l and A'-, hut the conclusions drawn arc 
equally applicable to problems involving any finite number of con- 
trol variables. In tlie two-control-variablc case the function can he 
plotted in two dimensions with the use of contour lines. For example, 
in Figure 12.1 the following simple function is plotted: 

V - (X. - D’ + (.V, - kf. 

The contours are drawn tlirough equal values of the outcome variable, 
V. Tlie minimum of the function falls at {h, k). If a function has 
more than one maximum (or mmimum), (licrc is no assurance that any 
of the procedures to be considered finds the greatest of tlicse. In ef- 
fect, then, these procedures arc designed to find focal maxima (or 
mmiisa). 


RANDOM DEMON 

Tliere are csseatially tno ways of selecting a random design for 
experimental optimization. In the first the range of values of .V| and 
A’a from which the eclection is to be made is specified. Then random 
selections of values of .Yj and X 3 arc made in pairs. In the second 
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probability that at least one of the raodran fombinattons out of n trials lies 
in the desirable sub-region is 

f* = 1 - (1 - a)“. 

This equation holds irrespectire of the number of factors. . . . 

Since the levels of the dilTcrent factors are independently assigned, methods 
of regre^on analysis can be applied to provide clues to the factors everting 
most influence. This approach must be developed further and compared with 
other methods before its utility can be assayed. 


FACTORIAL DESIGN 

Suppose that each of the scales along which and Xt are meas- 
ured is divided into equal intervals. Then the pairs of points can be 
represented as is done in Table 12.1. Observations made at the pair 
of a:-values represented by each cell constitute a factorial design. 
Such a design yields a set of observations which are uniformly spaced 
oa a flat surface, but when this pattern is projected on a curved sur- 
face those points arc furthest apart which fall oa the part of the sur- 
face with the steepest elope. The observations obtained from this 
design, like the random one, may be used to contour a three-dimen- 
sional surface, or to locate a region of relatively high outcome. This 
region can then be explored by selecting more closely spaced points, 
using this or some other experimental design. 

By now it will be apparent to the reader that many other types 
of experimental design can be used in probing for the optimum of a 
function. Cochran and Cox (1957, p. 36G) suggest rotatable second- 
order designs (wliich they discuss on pp. 344-347) and use of higher- 


Table 12.1. A Factooial Design 


Values of ,Vj 

Values of Xj 

Xu Its • • • Xin 

Xu 


Xl* 


Xu 
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order polynoiuinls. Which design is best will depend on the nature 
ol the function being expiorwJ. Consequentiy, by meiias of double 
(i.o., two-stage sequential) sampling an estimate of the form of the 
function can be obtained Itoni Hie firet sample and an appropriate 
design established for the next probe. Tlic logical extension of this 
principle is a sequential procedure in wJiicb we continuously move 
closer to the region of the optimum and explore each region in a way 
that is indicated by all the data available before each stage of ob- 
servation. These principles we will find incorporated in the method 
of steepest ascent. 

If the time between setting Hie values of the controlled variables 
and observing an outcome is large, sequential or multiple sampling may 
be impractical. The experimenter may not have access to the phe- 
nomena in question for a long period of time. If, for example, travel is 
required to make the observations, the cost or time of going back and 
forth may be prohibitive. 


THE SINCLE FACrOn PnOCEOtfllE 

This method, suggested by Friedman and Savage consists 

essentially of probing each controlled variable separately in the order 
of thefr estiioated importaoce. It may be necessary lo perfona more 
than one cycle through the set of controlled variables. The steps 
involved in the procedure are as follows: 

(1) List the controlled variables in the estimated order of their 
effects on the outcome. Let represent the most important, A'j the 
next most important, and so on lo the least important, A’». 

(2) Estimate on the best grounds available (wiiich may be a guew) 
the optimal combination of values of tlie controlled variables. Let 
tiiese values be represented by Xit, Xh, • • •, xji. 

(3) Fix the value of A'a at X* at x*i, and so on. Find the 
value of -Yj which maximizes the outcome, V. 

Obseivations at from three to five values of Xi arc required, de- 
pending OB how widely Xi ranees and bow well the region of its opti- 
mal value is known. For a discusrion of how to space these trials see 
HoteUing (im). 

If the experimental points straddle the ro.aximum value of V, fit 
a parabola to the data; for example, if 

r *= bo + biXi + bjX|*i 
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then, by use of the calculus, the maximizing value of A’l is found to be 

h 

“■-“-Si' 

If the experimental points do not straddle the maximum, further 
observations should be made until tlic maximum is contained. Then 
proceed as above. 

( 4 ) Fix Xi at ii5, leave Xa, Xt at the values Xai, •••, in, 
and repeat step 3 for X^. 

( 5 ) Repeat for the remaining variables. 

If at the end of the first round the differences (ija — iji), (^22 "" 
rji), (zm — iti) are small, the experiment can terminate. Other- 
wise conduct a second round, varying Xi at the fixed values Zzs, 

This procedure is illustrated graphically in Figure 12 , 2 , Here only 
two control variables are involved. The variable A’j is judged to be 
the more important, and X2 is estimated to be optimal at Zj}. Hence, 
starting at the point (Zn, Zn) we probe along the vertical at Zu 
we find the maximum at Now we probe horizontally at Z22 until 
we find the maximum at za. Then we probe vertically at Z12 until 
we find the maximum at 123. Finally, probing horizontally at zss 
yields no improvement, so we slop at the point (zij, Xss), which is 
estimated to be the optimal combination of values of and Xs- 

The more important (he coalrolied variables are (relative to the un- 


Xi 



FiovBB 122. Single-factor op- 
tirmzatiOQ 


X, 
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controlled variables) , the more rapidly this procedure converges 
on the optimal set of values. The uncontrolled set of variables, it 
should be rc-emphasizcd, are assumed cither to bo constant or to 
be varj'ing randomly about their mean. It these assumptions are not 
satisfied and if the uncontrolled variables ha%-e a significant effect 
on the outcome variable, this method can lead to a poor approximation 
of the optimal values of the controlled variables. 

It should also bo remembered that, if the function has more than 
one maximum, there is no assurance Uiat more than one will be found 
or that the optimum occurs at the one that is found. 

Jt can Ijc fchomj that the atlranfage of the single-factor procedure 
over simultaneous designs increases as the number of controlled vari- 
ables increases. The larger the nmnlicr of controlled variables the 
more rapid is the relative rate of convergence on the mavimum by the 
ringle-faclor method. Friedman and Savage (1W7, pp. 36.5^69) 
suggest a procedure for further accelerating convergence on the maxi- 
mum during the third experimental cycle if it is required. 

. . . After the second complete rounil it might be dcsimhic to move oiong 
the vector defined by the end points of (bo first two rounds, m^toid of vary- 
ing the variables according to the original pattern. 

In this suggestion lies (he underlying ides of (he ntethod of steepest 
nccc&t, whlcli Box and ^Vllson were to develop a few years later. 


THE METHOD OF STEEPEST ASCENT 

In 1951 Box and Wilson first prcscnlcfl a procedure for experimental 
optimization which ponnits simultaneous probing of all Ihe controlled 
vari.ablcs. In effect, the procedure esUioatca sensitivity of the out- 
come to each of tlic controlled variables and moves in tlic direction 
of estimated maximum improvement along all scales simultaneously; 
it does so by moving along each scale in increments proportional to 
the estimated sensitivity of the outcome to each variable. The sensi- 
tivity is estimated initially by the slope of straight lines fitted to the 
data obtained in a small factorial experim^t. Wiien acceptable linear 
fits can no longer be obtained, quadratic fits may be used and sub- 
sequently, if necessary, fits Involving incrcasinglj- higher-order poly- 
nomials. 

In elTect this procedure moves idong n diagonal which, on the basis 
of previous observations, is aimed at the optimal set of values. It 
makes adjustments of its aim as it approaches the target on the basis 
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Factor 


Tabix 12.2 
Factor Level 


-1 +1 


-Vi III — 0.5AA’» i|i + 0.SAXi 
A’s iji — 0.5AX* x*i + O.SAXt 


of obsci^’ed errors in the original setting. Since the single-factor 
method proceeds only along lines normal to the axes, one would ex- 
pect that this “diagonal” procedure would approach the optimum more 
rapidly. In most cases it appears to do so. 

The initial steps of the procedure can be illustrated by a case in- 
volnng two control variables, Xt and A'*. TIic steps are as follows: 

(1) As in the single-factor method optimal v.alucs of A’l and Xt 
are estimated or guessed. Let xu and x-i represent these values. 

(2) An amount by which each factor is to be changed is detensined; 
aA*i and 4 .T 3 . These amounts should be large enough to show effects 
on the outcome hut not so large as to permit the possibility of much 
curvature of the functian in the interval between values separated by 
these amounts. A poor choice of aXi and AXs can be corrected later, 
but a good first choice reduces the number of observ'ations required to 
reach the maximum. 

(3) Levels of experimental factors arc chosen as indicated in Table 

12 . 2 . 

(4) These values are arranged into a 2 X 2 factorial design, and 
observations art taken at each factor eombinatioa. The results are 
tabulated as shown in Table 12.3, in which v represents the observed 
values of the outcome. 


Tabix 12.3 


* Xii 

Xa 


XilCj 

XrtC, 

1 +1 

4-1 


+V++ 

4-P++ 

2 -I 

-i-l 

V-+ 


4-p_+ 

3 -1 

-1 

r 

— c 

— p _ 

4-=n 4-1 

-1 

*•+- 


-p+_ 

Sum 



2>nr. 
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(5) A linear fit to Uic datft is obtained in the form 
r = 6o + 6iA'j + ij.Yj, 

nhere 



(0) Now probing of A'l and Xt begins by changing their values by 
incrementa proportionate to I>i and b*. For oxarapfe, if A'j is changed 
by increments la equal to K aA'j, then X, is changed by an increment 
/j, where 

0| 

/, 

h 

The eclcelion of K is a matter of judgment. If it is too large, the 
optimum may be passed. Consequently, A' is usually made conserr- 
flUvely amall. An observation is made at tlie values xu + /| and 
3^31 + /*. The result is compared with tlic value predicted by the 
iiocar equation [step (5)]. As long os the observed and computed 
values are relatively close together, further obsctv'ations are made 
for additional increments. When they diverge significantly, a new 2‘ 
factorial experiment ft » number of factors) is conducted at the 
last point. If the new linear fit is good, the procedure is continued 
until cither 

(а) the Vnesr fit is good Imt the coeflkients t, are all small, which 
indicates that a plateau and hence a masimum has been 
reached; or 

(б) UiC linear fit is not good, in which case the experimenter pro- 
ceeds to quadratic fitting. If the initial estimate of the loca- 
tion of the maximum was good, the first linear fit may not be 
good and the experimenter may have to proceed directly to a 
quadratic fit. 

We will not go into the detiuls of the quadratic phase of the pro- 
cedure. These may be found in a number of the references given at 
the end of the chapter, for example, Da^es (19SG). In some instances 
the experimenter may hare to resort to higher-order fits lhaa the 
quadratic. 
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In the method of steepest ascent, as in the single-factor procedure, 
there is no assurance that in functions invoU’ing more than one maxi- 
mum the optimum has been obtained. 


COMPARISON OF THE PROGEDURES 


There is no n-aj’ of making a completely general comparison of 
these procedures Their relative values depend on the amount of prior 
information available and the nature of the surface to be explored. 
Brooks (1955) did make a comparison of the four procedures for four 
very simple two-dimensional (ie., two-controlled-variable) surfaces. 
He equated the procedures by allowing only the same number of obser- 
vations in each. In this case tlie true maximum outcome was equal 
to 1.0. By making repented trials with each procedure, he computed 
the average estimated maximum outcome yielded by each method. 
The results are shown in the following table: 


ilfdliod 1 

Single-factor 0.990 

Steepest ascent 0.993 

Factorial 0.955 

Random 0.902 


Svrjau 

2 3 4 

0.9S4 0.928 0.DS4 

0.089 0.979 0,985 

0.977 0.927 0.07G 

0.911 0.913 0.036 


The method of steepest ascent was uniformly the best, although in 
all but the third case it was only slightly better than the single-factor 
procedure. As one might expect, the random procedures was uni- 
formly poorest. In the third surface, where the single-factor pro- 
cedure performed least well relative to the method of steepest ascent, 
the factorial procedure performed about as well as the single-factor 
procedure. 

General conclusions carmol be drawn from this test, even though it 
may seem to conform with what one would generally expect. It should 
be remembered, moreover, that all the surfaces explored in this com- 
parison had only one maximum. 


CONCLUSION 

Although the developjnent of procedures for experimental optimiza- 
tion is still in its infancy, some of these procedures are quite useful. 
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To bo sure, tlie rnnRC of iUuations in which application of these pro- 
ctxlurcs is justified is limited, but it h nevertheless extensive. The 
principal applientions to dale have lx.-cn made in the study of such 
processes a.s occur in rhcinical production, metal production, and 
some areas of ngricultiire. In general the proccflurcs have been found 
most npplieaiilc to geogruphieally eonfined phenomena TJie opjior- 
liinities for exjwrmifntal oplimizution are liest when the obsers-ations 
r.an Ite made under such highly controlled conditions as can be creatciJ 
in n lalHjraloiy, a pilot plant, or a confined work area. 
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TESTING AND 
CONTROLLING THE 
MODEL AND SOLUTION 


INTRODUCTION 

Is the model an adequate representation o( tlie phenomena under 
study? The procedure by which an answer to this question is sought 
constitutes the testing of the model. This aspect of research is not 
separated in time from other phases of research but continuously inter- 
acts with them, particularly with the construction of the model. UTiile 
the model is imder construction, it should be subjected to continuous 
testing. Determination of whether the model contains the relevant 
variables and constants, whether these are evaluated appropriately, 
and whether the functional form is correct are all aspects of testing 
as well as of model construction. 

The testing of the model that is conducted simultaneously with its 
construction, however, is usually directed toward parts of the model 
rather than toward the model os a whole. Once a model is completed, 
there usually remains the need to test it as a whole. It is this phase 
of testing with which we are primarily concerned here: the final over- 
all test of the adequacy of the model and the solution derived from it. 

In a very fundamental way every model is a predictive instrument. 
It predicts a value of some outcome as a function of the values of a 
specified set of variables and eoostsnts. Hence, testing the model 
as a whole consists of testing its ability to predict. Since prediction 
is itself a procedure for estimating future values of a variable, test- 
ing consists of determining the principal characteristics of the esti- 
392 
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mates which are yielded; bias and reliability. The procedures for 
testing the model as a whole, tl)ercfojT, are essentially the same as 
those involved in the evaluation of any estimating procedure. 


TEST FOR DUS 

The model should be compared with the real j>heaonienon by com- 
paring values of the outcome predicted by the model witli actual out- 
comes, where these can be deterndned uith little or no error. This pro- 
cedure may he conducted either retrospectively or prospectively. 
That is, for each set of instances of tlie phenomenon in question, an 
outcome is predicted by use of the model, and these predictions should 
be compared with the actual outcomes If past instances ate used, the 
test is retrospective; if future instances are used, the testis prospective. 

In the use of past data for evaluating a model ao ciTective evalu- 
allot) caocot be obtained tf the model is tested against the same data 
which were analyeod to obtain estimates of the parameters or the 
functional form. For this reason it is generally advisable to divide 
available past data into two parts, one to obtain the necessary infor- 
mation for evaluating tJie variables and functional form of the mode], 
and the other for testing. If tlie model stands the test well, then all 
the data can be recombined to obtain more precise eatlnates of the 
variables and the functlonol form. 

It is particularly important to divide the data in the way indicated 
if the model is empirically derived, using regression or correlation 
analysis. In most cases an equation which is derived from a set of 
data can be expected to predict those data reasonably well. The 
critical question is whether it will accurately predict events not 
covered tjy the data from which it is derived. Models have an un- 
pleasant inclination not to bold up beyond the period from which 
they were derived. 

In testing the model the difference between llic predicted outcome 
and the actual outcome should be determined for a series of unique 
events. Let d( represent these observed differences. By use of these 
observations two questions can be answered: is the model a biased 
predictor of the phenomenon in question, and how reliable a predictor 
is it? 

The model would be unbiased if the true average difference between 
the observ’pd and actual outcomes, D, were equal to zero. Therefore, the 
test for bias consists of testing tboh^wtbeas; 

Ny.D - a 
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This test consists of detcrmimnK, if llo irero true, the prohabilily tfiat 
we would obtain the ob«cr%'c«l nvrrage tloviation of oljson'wJ outcomes 
from predicted outcomes: 

-Erf.. 

n 

where n is the number of ob-'crvations made. Standard statistical 
procedures may be used for tlds purpose (c.g , tJie /-test). 

If the hypothesis //a is rejected on the basis of the test, it is neces- 
sary to adjust tiie model so as to remove the bias. This may bo nc- 
corapli'hcd simply by adding or sulilrocting a constant equal to the 
estimated bias. Such an adjustment, however, docs not add to our 
understanding of the phenomena unless we can determine what this 
constant represents; that is, what relevant aspect of reality is missing 
from the model or what irrelevant aspect is included. An error in 
the functional form of the model may also produce a bias. The ad- 
dition of a bias-correcting constant which is derived from the part of 
the data that is u«cd for testing purposes makes this portion of tlic 
data unfit for further testing of the model, lienee, it is important to 
determine wimt causes the bias of the model and not merely to add an 
empirically derived correction factor. 


TESTING FOR RELIABILITY 

Tlie reliability of the model can be measured by estimating the 
variance {or some other appropriate statistic) of the deviations of (he 
obsen’ed outcomes from those that were predicted. There are no 
simple criteria for determining whether the variance is too large; that 
is, wliether or not the model is sufficiently reliable. Practically, the 
important question is whetljcr the model yields more reliable pre- 
dictors than alternative models of the same phenomenon. If it does 
not do so, it does not necessarily follow that it is not good enough. 
A more reliable model may be more costly to use than the less reliable 
one; and consequently, when the “cost” of prediction is taken into 
account, the less reliable model may be preferable. The situation 
here is the same aa that iavolved. whacewee an. a^^c(4xiTOa.tAon. pence- 
dure is used. That is, the determination of the adequacy of a model 
requires consideration of the way it is to be used and the costs in- 
volved. 

As before, let V represent the true value of the outcome variable 
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find I' tlip vfifuc r.>-iiiuat«l hy tlic model. Tiicn let a,- represent the 
true variance of tticso cstinwites and *,= the estimate of a,*. Let Ct 
JTim*?ciu the “cost" of i)np.aniiK n prediction usjiiig tiic raotlel. Now 
it is noccNsarj* to dMcmune Hit- e\prrled post of the in.'icfiiracy of tfic 
predietiona. We simll examine the proeeilurc for doing so in a mo- 
ment, hiit for the time heiiig let pj reptn'ent tJiis cost. Then alterna- 
tive modeling u-clmifiues can be compared relative to the sum of 
these costs; that is, the best model is that one which minimizes the 
Slim, Cl 4- r-. 

Before engaging in tlie stejw involved in testing the reliability of 
the jsjodc), it is frwjurntJy tlesiraWc to have the model thorougld}- 
srnitinited by other rweareheib who an* knowledgeable in the area 
Invoired. Such organized critifal sessions arc sometimes referred to 
as “murder boards,” since Uicir objective is to “destroy" the model 
Lvnluntion of tiic criticisms and suggCHtions generated by such n re- 
view Is, of course, the ultimate responsibility of the rMcnrchers in- 
volved. Despite the obvious desirability of such reviews they are 
l>Dt used as much as one would i'.Y|>crt. This )jas it'd one neademie to 
observe that many rcseareiicrs would rather publish (and hence pub- 
licly displayj their errofs than reveal them privately to their col- 
leagues. 

Determination of cj, the expectwl cost of error rwulting from use 
of the mo<kl, is usually extremely difTicult in so-enlled pure research, 
where the uses to which the model is to ttc put may not be known 
or rannot rrodily bo evaluated. If this is the case, tlie scientist must 
rely on his judgment nnd experience. In applied research, however, 
(Ilfs cvniu.ation can be fwrfbrmcd explicitly, since a solution to n 
problem is involved. In fucIi an evaluation the model und the solu- 
tion derived from it are treated as a unit and not as separate entities. 

I/Ct di now represent the deviations of Uie actual outeomc of the 
phenomenon from that prcdicleil by application of the solution. That 
{y, it V, represents the true opliin.al outcome and f« the outcome ac- 
tually obtained by use of the solution, tlien in nny specific instance 
(t'a — r,) Is eijual to d. This demlion would be tuipressed in value 
units. The cxpcelcd value of di would then bo the c.xpectcil “cost” 
of error in the model and its solution. Since the true optimal out- 
come, v„ is not known, such an idealized evaluation cannot be per- 
formed; that is, r, is a standard of performance which we cannot 
specify. In situations in which we do not have a standard against 
which alternatives can be evaluated we must generally compare them 
with each other. Hence, luting a model nnd the solution derived 
from it, in a problem context, involves comparing the performances of 
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two or more solutions. In most problem contexts the comparison can 
include the decision procedure used by the decision maker before the 
newly derived solution was obtained and adopted. 

Ideally this comparison should be made by solving the problem 
using the nitemative solutions under a set of identical circumstances 
and by comparing the differences in performance. Such a compari- 
son can soraeCinics be roaiJo by use of test areas, pilot operations, and 
so on. When it can, the average difference in performance is the rela- 
tive expected cost of error of llie least cfTectii’e solution. Then, by 
taking into account the co«t of obtaining the solution of eacli proce- 
dure in a specific sitimtion, the sum of the costs for each procedure 
can be obtained and n comparison made. 

In many instances such a rontrolled comparison between two meth- 
ods of solution cannot he made. Such comparisons nre usually not 
possible, for example, in problems involving decisions of managers of 
industrial, governmental, or military enterprises. In such instances 
we have to approximate a comparison. 

Comparison of a proposed method of solving a problem with a pro- 
cedure that has been in use may be made retrospectively. It con- 
sists essentially of dotermimng what the perfomiancc of the proposed 
procedure would have been in instances in which the oilier procedure 
was actually used. The dtfRcuUy inherent in such a method is that a 
prediction of what would have happened if the solution had been 
used must be m-idc by means of the model which is under test. The 
results, then, yield a comparison which would be perfectly accurate 
only if tlio model Itself were a perfect representation of the problem 
situation. This procedure is biased in favor of the solution derived 
from the model. Yet it is by no means useless. 

What such a comparison actually yields, as noted, is a comparison 
of performance of alternative procedures of solulton if the model on 
which one is based is in perfect correspondence with reality. If the 
difference observed between the average performances is small, one 
would have good reason to doubt the superiority of the proposed solu- 
tion. If it is large, one can then raise some relevant questions which 
help to support or destroy the apparent superiority of the proposed 
procedure. For example, if it is assumed that the only errors in the 
model are ones iavoliung evaluation of variables, it is possible, using 
tlie data from the comparison, to determine by how much the esti- 
mates of uncontrolled variables would have had to be in error before 
the proposed solution would perform less well than the procedure 
with which it is being compared. Such a procedure can be illustrated 
by using the familiar simplified model: 
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= + (1) 

Suppose tfiat we must find the optimizing value of A' in this model. As 
we have showm previously, 



and hence 

r« = 2Vr. (3) 

Suppose that an estimate of Y,j/ = 25, is obtained. Then the estimated 
optimal value of A* would be 

la = I/V25 => i. 

If r is actually equal to 25, then 

1’. = 2V25 - 10. 

Now suppose that under these conditions another decision rule yields an 
actual performance equal to 14.6. Now we can ask, when 1' » 25, for 
what value of j/ is — 14.5, where 

r, ss - 7 * + (^) 

Vy 

To answer this question we substitute in (4) and solve: 

14.5 = 25/Vp+ 

M.SV'J = 25 + } 

»-14.5Vf + 25 = 0 
(vV - a)(vs - 12,5) - 0. 

Hence, 

2 

y = 4 

Vy « 12.5 


I6G25. 
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Tiien, if }' actualiy were equal to 25, we would iiare to underestimate 
by (25 — 4 =21) or overestimate by (156.25 — 25 = 131.25) to do 
as poorly as did the alternative-decision nilc. 

Such computations can be repeated, and by so doing we can esti- 
mate a distribution of "equalising estimation errors.” Now we can 
inquire as to whether such a distribution of errors is likely to occur, 
using a specified estimating procedure for Y. If not, we are relatively 
sure that the proposed solution yields an improvement even though 
we do not know precisely how large tfie average improvement will be. 


CONTKOL OF THE SOLUTIONS 

Control of a model consists of setting up a procedure for detecting 
significant changes in either (o) the values of the parameters, or (b) 
the mathematical form (structure) of Uie model. The model, the solu- 
tions derived from it, and the control procedure together constitute a 
self-maintaining decision-making system. Systems wluch, on the basis 
of information feedback on their performance, adjust their parame- 
ters so as to improve their performance, arc called adaptive systems. 
Those which are also capable of modifying their structure so as to 
improve performance arc said to be sdhorganizing. A dynamic self- 
maintaining, decision-making procedure, then, is an adaptive and 
self-organiiing system. 

If the model and/or the solution derived from it are to be used re- 
peatedly, it may be possible that the values of the uncontrolled vari- 
ables will change during the inten-al between applications. Conse- 
quently, if it is known that such repetitive use of a decision rule will 
be made, it is desirable to check the values of the relevant parameters 
periodically in order to make sure that the proper values are being 
used; and, if not, to adjust them. Such a procedure we refer to as 
control. Periodic checking of the functional form of the model is 
also a control procedure. 

If the parameter is not statistical in character (e.g., the charge for 
an hour's labor), no particular problems are created by the control 
process. Complications arise only when the parameter involved is sta- 
tisiicsl in ehsractcT (e.g., aa average vakie, a percentage, or a prob- 
ability) and where the values arc estimated on the basis of a sample- 
The control procedure in such a case can he characterized as follows: 

(1) A significant change in the value of a variable must be defined 

(2) The design decisions involved in a control procedure are 
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(o) tlie freqiieuoy of control checks; 

(6) the tj-pe of sample selected; 

(c| (lie sue of the sample; 

id) the test for significance of change; and 

(e) the significance leve^ at which the test is to bo conducted. 

(3) Ideally, these decisions should Ihj made in such a i\ay as to 
inininiixc the sum of the foJJoiruig costs: 

(а) tlie cost of making the ohservations; 

(б) the cost of the t<st; 

(c) the expected cost of ^pc I error; and 

(d) the expected cost of type II error. 

The problem is similar to designing a lest for a hypothesis with an 
additional eoinplication arising out of the fact that the tost is to bo 
performed repetitively, If we knew the distribution of changes of tlie 
true value of the parameter Involved with respect to time, It would 
be possible in principle to opliinUc the frequency of control decks. 
Eince this inforroation may not be available, it may be necesiary to 
select a frequency of control chocks on other grounds. 

There is usu.ally little or notliing to be gained by cheeking the val- 
ues of parameters any more /requeotiy tiiao the solution is applied 
(i.e., a decision is made). The only point of chcckiDg the value of a 
parameter Jess frequently (e.g, at every second or third application 
of the Bolution) is tliat the parameter is extremely stable or that the 
cost of the control check Is vciy high compared to the cost of possible 
error. The latter condition is mtlicr Mrc. Consequently it is gener- 
ally reasonable to run a control cheek on each parameter before each 
application of the decision rule. 

The remainder of tlie control problem is Identical with tliat of test- 
ing a hypothesis, but it is necessary to specify the hypothesis to be 
tested. We wmdil want to change our estimate of tlie parameter only 
ivJien the cost of fo doing is Jess than the expected gain in perform- 
ance. It is necessary, therefore, to determine the magnitude of the 
least change in the parameter which results in a loss equal to the 
cost of the adjustment. The cost of the adjustment will vary' from 
problem to problem, tlcjxmding on the complexity of the solution and 
on such otlier factors as wliether the process Involves ii computer pro- 
gram or smipfe fiantf computations. In many cases whrch nmrlfv 
hand computation, tlie cost of changing Uic value of a parameter Is 
negligible. Where a computer program would have to be materially 
changed, hoivever, tJio cost could be quite l.irga Once this cost of 
adjustment, f,,, has been determined, ta*!!: Ib.Tt remains is to find 
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for cadi uncontroDed parameter how large a cliange in its true value 
woxild be reciuired before the loss relative to the outcome variable is 
equal to Ca. This procedure can be illustrated by use of the model 
given in equation (1). 

Again assume that 1’ = 25 during the last period. Then the op- 
timizing value of X is 1/V25 or and the minimal V b 2V25 or 
10. To find tiie critical upper and lower values of Y, Yg' and Yt,', 
we proceed as follows. 

The cost of error of using Y =■ 25 when Y is actually some other 
value, is given by the cost of error (C), which is 

C = ;^+ V?-2v'r. (5) 

In Ihb case 

C = ^ + 5 - iW'. (6) 

s 

Suppose that the cost of clianging the value of V is 0.2. Then we set 



C = y+5-2VT'-0.2 

OP 

Y' - wW' + 24-0. 

Tliercfore 

and 

^'/r - 6)cVF - 4) - 0 


= or le. 


Hence, if Y changes in a positive direcUon by more than (36 — 25) 
or 11, its value siiould be changed. If it reduces by more tliaa 
(25 — 16) or 9, its value should be changed. (The lack of symmetrj’ 
reflects the asymmetry of the function which gives V.) Hence, in the 
control process we must check the hypothesis; Id^Y^ 36. 

To illustrate how significant changes can be determined in the two- 
parameter case, consider the following decision model: 

Tlie optimizing value of X is found by Uie calculus to be 


( 7 ) 
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and the optimal perfonnance is 




Now suppose tlmt Ihc cost of a change in the solution is 2, and that 
the last values used /or 3’i and Ys were fft S and = 2. The cost of 
not changing the solution when these \*nlucs actually do cliutjge is 
given by 


* ‘ 'iui ^ V'tfa/Vi 


- 2\/k,V2 


( 10 ) 


Now, if C is set ecjual to 2, we can detormino for various values of Ki 
what the corresponding valuea of I'j are. For csamplo, let I'l - I, then 



- } + ar, - 2 vT^ 

1. + - i - 0 

(VT^ + t.5)(Vl^ - 0.S) - 0 
J’j - i and —2.25. 

Similarly we cun compute other values of Kj and obtain results ns follows: 


I'l 

1 -2.25. 025 

4 0. 4.00 

0 023 . 625 

Ifi IJX), 0.00 

2.7 223, 1225 

30 4 . 00 , 10.00 

etc. 

Fmai this a plot esa be cbtsiaed mei as is shoiro in J^i^re 13J. 

Once the significant changes have been determined, the hypothesis 
to be tested in the control check is that tiie change in value of Yf 
since its last use does not exceed the significant amount 
As the number of parameters in s model increases beyond two, the 
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number of possible combinations of tlicir value wliich can constitute 
a significant change becomes too la^e to be manageable. In such 
cases it is generatiy preferable to determine the effect of any ap- 
parent combination of changes when they arise rather than to try to 
set up criteria in advance. 

Once significant changes in the value of a specific parameter in a 
positive and negative direction have been determined, a control chart 
— resembling those used in statistical <iuality control — can be pre- 
pared and maintained. Such a chart facilitates the apprehension of 
trends or other types of nonrandom ffuctuatione in the value of the 
parameter. If more than one parameter is involved, the bounds are 
variable, as has been sbomi. Even so, it is helpful to prepare a con- 
trol chart for each parameter separately, assuming reasonable values 
of the other parameters. The resulUng charts should always be stud- 
ied collectively. 

As in quality control, when a significant change occurs, an effort 
should be made to find the reason for it, the assignable cawe. If 
such a cause is found, it may reveal whether or not the change can 
be expected to persist. In ei&er case it would be desirable to intro- 
duce the identified cause of the fluctuation into the model to assure 
future awareness of significant changes due to its behavior. For ex- 
ample, in a problem coataining “c^nge in eales per month” of .•» 
pharmaceutical as a parameter, a control chart revealed that this 
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parameter had periodically gone out of control in the past, and had 
fjuickJy come back into control. Further study revealed that in each 
such instance a price increase had been announced for the next month, 
By inserting “change in price for tlic next month” into Die model as 
a parameter, giealcr control over the outcome was obtained. 

If an assignable cause cannot be found when a parameter goes out 
of control, close observation siiould be maintained to determine 
wlietlicr the variation was a random phenomenon, or whether further 
and more inleneivc feareli for tlie cause should be carried out. 


CONTROLLING THE MODEL AS A WHOLE 

As an alternative to continuous testing for change of each aspect 
of the model, such a test can be performed on the model as a whole. 
At the beginning of this chapter a procedure was described for testing 
the adequacy of the model. Once the e-tpected error (eero when the 
model la unbiased) and the standard deviation (or some other suitable 
measure) of tlic error has been determined, statistical control procC' 
ilures can be applied to these errors. If the errors display nonrunciom* 
ness (c.g., trend) or an unlikely magnitude (under tiie assumption of 
no change), tlie individual aspects of the model can then be examined 
to detcrrauie what specific error caused the observed deviation. Then 
appropriate adjustments can be matlc if warranted. Tlie errors can 
he measured in terms of the output variable; ami hence, when the 
value of the error in question exceeds the cost of adjustment, an ad- 
justment is justified. 


CONCLUSION 

Even where tiie mo«t complex models arc involved, a considerable 
amount of control c.an be obtained by relatively intuitive application 
of even elementary statistical tccliniqucs.* These and more sophisti- 
cated procedures are least often used where the mathematicaf sopliisti- 
c.ation of the model is greatest, precisely where the greatest gains are 
to be made by means of conlrol procedures. 

Finally, it should be obseiwed that the model of a problem eitua- 
lioi! is a theorj' which attempts to explain the phenomena involved. 

•Scercfcronrcs «l tlic cnit of Iba rhsplcr for imwc dctcilnl ilincucion of lliw 
Icihniqucs. 
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The predictions and solutions derived from the model arc deduced 
consequences of the tfieorj*. We have, in cfTcct, been discussing ttcorj' 
testing in applied problems. Tlicory testing in pure science is, there- 
fore, the same in principle as the procedure we have considered here. 


BIBUOCn.\FI{Y 


AcUoil, R. L, "The Concept and Ejeitnc of Conttol m Operations Research," 
Proccrdiap* o] tJie Fir»{ /n(enw(i<ma( Conjerence on Operation* Ruearrh, 
Baltimore, Operations Rcscarrh Society of America, 2cd cd, 1957, 2<1-13. 

Duncan, A. J., Quality Conirof and Induatr^ Slatulif*. Chieajta: Richanl P- 
Irwin, I0S2. 

, "TTic Economic Design of 5i Charts Used to Maintain Current Control 

of a Broces,” Journal of (he Amcncon StatutieQl Socutv, fit (1956), 23^313 

Grant, B. L., Stotuticai Quality Control Kew York; ^IcGraw-nill Book Co, 
1952. 

Juran, J. M. (ed.), QualUn Coj>trot Handbook. Kew York: McCraw-HiU Book 
Co, tow. 

Littauer, S. B, ‘‘Techoole^cal Stability ia ladustrial OperatloBS," Tremaetiorti 
of (Ac Ntv Yorli Academy of Science. Ser. II, IS, No. 3 (1950), 66-73. 

— "The Development of Sututkal Quality Control b the United States," 
American Statialicvsn, 4 (1950), 14*20. 

Shevhart, Vi. A, Statuticof Method* from the Victr n’nt >/ Quality Conlru^- 
Wuldogton: United States Department of A^nihute. 1939. 

Uppett, L. n. C, TccAnoloficat Applicationc of Stetiitice. New York: John 
Wiley and Sons, 1950. 



14 

IMPLEMENTATION AND 
ORGANIZATION OF 
RESEARCH 
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the players have been graduate students in management who have 
had courses which equip them to construct the model and to derive 
the solution. 

One member of the team is a “planL” He knows the solution to 
the game. After a number of plays of the game by a team — during 
which few teams have yet made an effort to construct a model and 
derive a solution — the “plant” pretends to discover the solution and 
suggests its adoption by the team. As yet, only a very small propor- 
tion of the teams has accepted the suggestion. 

The experiments are directed toward finding tlie conditions under 
which a suggested solution to a problem will be accepted and imple- 
mented. Using what is "known” about these phenomena — ^most of 
which is obviously folklore — the experimenters have not yet been able 
to find a way to present the solution to teams so that it has some 
significant chance of being accepted. 

The moral of this extremely informative and provocative experi- 
ment is that at present we do not know enough to assure acceptance 
of research results and their implementation under what appear to be 
very simple and favorable conditions. 

The fact that the experimenters have had less success in obtaining 
implementation than most researchers cfatm to have in practice sug- 
gests either that the game situation is not as simple or favorable as 
it appears, or that implementation of research results in practice is less 
frequent than one would suppose. Nevertheless, until the researcher 
can solve the implementation problem in situations that are completely 
under his control, he cannot pretend to have substantial knowledge 
of implementation in situations which are hirgcly out of his control. 

Tim nESEAPCUER’S RESPONSIBILITY 

From the Renaissance up to the recent past society has discouraged 
entry by the scientist into the realm of practical affairs. Even today 
the burning of Bruno and the punishment of Galileo are vivid in the 
scientist’s mind. Past persecution and pressure have bred a belief 
that the scientist is responsible only for producing knowledge and not 
for the way in which it is used. Since the development of the atomic 
botnb, however, a gradual transformation has taken place in both the 
public and the scientific attitude toward the involvement of science in 
public affairs. Tlie use of science in solving practical problems has 
been increasing at a considerable rate, and, in fact, many obsen’ers 
have pinned the possibility of continued social and economic progress 
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on just such involvement of science For example, in Ills book, The 
Scientific Attitude (1948), C. H. Waddington has written fp. vii); 

Tho aim of this book is to point out some of the contributions which the 
scientific attitude of mind can make to the creative tasks of social reorgani- 
sation with which the world is faced. By this I do not mean merely an 
enumeration of new devices, new soune<i of power or techniques of produc. 
tion. The fundamental proldems of to-day lie far more in the sphere of ideals 
nnd valiies~in the spiritual sphere, if you are not afraid to use that term— 
tlian in the technological. We already have all, or at least most, of the tech- 
niques we need to provide a decent civilized life for everybody. They are not 
yet producing tliat result, partly of course because the organisational ma- 
chinery has been smashed in large part* of Europe, and partly because the 
old political and ecooocuic tnachine was unsuitable, directed to the wrong 
ends, and in obvious need of thorough overhaul, which unfortunately many 
people were not yet ready to give it We now b.ave to decide for what pur- 
poses the new organisation shall be designed And to the discussion of these 
social ends, science has a good deal more to contribute th-an has been recog- 
nized hitherto. 


Recommendations 

There are still some persons in and out of science who feel that the 
researcher sliould simply determine the consequences of actions of 
various sorts and not recommend action or take any responsibility for 
lliese consequences. The fact is, however, that with the development 
of an adequate methodology of problem sohing, a solution carries 
with it the prestige of science and is jDterpreled ns o recommendation 
whether or not the scientist wishes It to be so considered. 

To be aure, all solutions to problcma are conditional: they depend 
on the adequacy of the representation of the problem situation by the 
model employed. That is, at beet the solution derived from the mode] 
is a solution to the relevant problem only if the mode) adequately 
represents the problem situation. The adequacy of the model is clearly 
the responsibility of the researcher. If the model is less adequate 
timn the researcher would wish, but as adequate as he is capable 
of producing, he should see to it tliat the decision maker is aware 
of the limitations of tlie model. The ultimate responsibility for 
decision making, of course, lies with the decision maker, who must 
evaluate the research and its results. He may also have to supply 
judgment to cover those aspects of tfte problem which couM not he 
covered by the research. TTic dccnion maker’s ultim.'ite rcsponjibil- 
ity, however, does not relieve tlie researcher of any responsibility for 
consequences of action which follows foglealfy from the research 
findings. 
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The fact; is tliat decision makere who sponsor researcli into their 
problems expect recommendations from tlie research that they sup- 
port. They will not continue to support applied research which does 
not yield recommended solutions to their problems. 

Implementation 

Even some of those who would accept the necessity of the research- 
er’s providing recommendations will deny his responsibility for cariy- 
ing them out. Such a position ignores important practical and scien- 
tific considerations, liie principal practical consideration is that a 
decision maker who sponsors research usually expects and requires 
the scientist’s participation in implementation and frequently selects 
scientists for applied research on the basis of their ability to follow 
through. To the scientist who wants to study real problems but not 
to manipulate the real world, this expectation of decision makers may 
not seem to be a potent reason for allowing himself to be so used. 

There is, however, an important scientific consideration involved in 
the implementatioD issue. It is only through implementation that 
many research conclusions can be tested because an increasing pro- 
portion of research is concerned with phenomena that cannot be 
brought into the laboratory. For euch research, application of its re- 
sults in the "real" world can provide the strongest test of their va- 
lidity. If implementation is in the hands of nonscientista, modifica- 
tions in the research results are inevitable, and they are likely to be 
made in such a way that the usefulness of the application as a test of 
the research is destroyed. Modifications of problem solutions are gen- 
erally required in practice because of changing conditions, but such 
amendments should be made under the direction of those who are thor- 
oughly familiar with the way the research was conducted. 

The researcher’s role in implementation should be much the same 
as the architect’s in the construction of a building he has designed: 
that of active supervision. Just as the architect is the only one who 
knows enough about a building to supervise its construction ade- 
quately, so the researcher is the only one in a position to super\’ise 
implementstion of most researeh results. 


TIIE PLAN FOR BfPLEMENTATION 

Planning for the implementation of research results should begin 
when the research itself begins; it should not wait until the results 
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fire oLlainoti. SpccififaHy, tiie Icchmcal abilities of those who will 
use the results and the facilities at their dtsjiosal should ho taken into 
necount in tleterniining the form and nature of the rcsearcii results 
H'Jiieli should tic .'ought U would be foolish to expect a clerk to solve 
an equation requiring the culciilu' of variations; a nomograph or a 
table may he neceasan.-. Rut u iiuinograpit or a table may be able to 
provide only very approxiiiMte 'viutiom to wjuations. An approxi- 
mation wliicli is used, iiowcver, will prmlucc better results than an 
exact solution which la ignored. 

The extent to which reaearcli results are distorted in use, or the 
likelihood that they will l>e used at all, depends critically on the 
extent to which those who will use these results understand them and 
can cari^’ them out. Hence, it is icnportanl to formulate and solve 
a problem in such a way as to yield a type of solution which is oper- 
ationally— not merely logically— fensiWe. 

In order to assure Uiat the research results are carried out as in- 
tended, it is necessary to develop a detailed plan for tlieir implemen- 
tation. This recfl is generally acknowledged where the action ulti- 
mately to be taken is to be performed by a computer. In such a sit- 
uation the researclicr recognises bis responsibility for developing a 
program for the computer. What Is not as well recognised is that 
almost as detailed a program is required for human operators. It Is 
necessary to specify e.xactly who is to do what, when they are to do 
it, and how. The who and u-hen can normslly be shown on a flow 
chart which indicates the way that the relevant operations are to be 
conducted. The wAaf requires detailed instnictions in terms of oper- 
ations that Can be performed by the kinds of people involved. 

An example of implementalion instructions 
To illustrate the kind of detail required in such instruction let us 
quote from a report of research on the budgeting of research and de- 
velopment activity in a business firm. Tbe details of the study and 
the meaning of the instniclions are not relevant here. \^T3at is im- 
portant is the jfi/lc of the instmcUons. The instructions quoted yield 
a graph which, when used in combination with others for which in- 
struclioDs are also providctl, produces the budget sought. To aid 
those who will use the instructions in seeing the whole computational 
procedure the diagram shown in rigwre 14.1 is also provideil: 

(1) From. Table 1 selecl s value of tbe factor (F) eorresrpoeding to cur- 
rent market share. . „ . „ 

(S) ScJ«t s trial value o! Ciannlali« Product Research E-xpendatures. 
f3) Calculate the product of (1) and (2). 
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r. 141. SchemRtic diaerem of computationgil pToecdure, part I. 
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(4) Prpiiare ad estimate of next ycar’« OH Products Bales 

(5) Using t!)e estinutc !rom (4) and tl» trial value front (I), detcnnine 
from Figure lOi the Total Riles. 

(6J Using the Total Sales from (5), dptermenG from Figure lOh the Gross 
Value of riant. 

(7) Select a trial value of net return on Gross Value of Pl.int from a pre- 
seJectetl set ef tri.-jl vafues. 

(S) Jfultiply Use results of (6) and (7) 

(0) Add the renilta of (3) and (8). 

(10) Subtrart the result of (»> from llwt of (3) Call the result .1 

IlejKiljlion of tliesc ten jfejw for mrioiM trtil values of Product Researclj 
Expenditures yields Figure J lo. 


(ID 

( 12 ) 


(ID 

( 15 ) 

( 10 ) 


Select a trial value of Cumulathc Proce/!s Rescareli Expenditures. 
AfuJuply (he value eelecieil from Table I in step (2) by the trial 
value selected in (11). 

Using the value of CumiiLativc Process Ue«e.irch Expenditures re- 
lecled in (II), determine from Figure IGf (he value of (he intermedi- 
ate variable i. 

Uang the \‘nlue of Total S.iles from (5), dctennine from Figure 10J> 
the vnluo of the interronliatp xximblc *. 

Multiply the value of 1: obtained in (13) by the value of 7 obtained 
in (14): 

Add ik (15) to the rcMilt of (12). Call this result B. 


Repetition of the last six steps for various values of Cumuhilive Proecss 
Reemreh Expenditures, grouping the remits l>y values ot r obtained in (14), 
yield* a jgurelilce Figure Jlh. 

(17) For cacb (rial value of Cumulativo ProJiict Research Expenditures 
used in (1) list the corresponding value ef * used in (14). 

(18) Using the value of Curoulafivv Product Re«.arch Expenditures from 
(17), determine from Fgure llo the corresponding value of A. 

(19) Lei the value of B equal that of A obtained in (IS) 

(20) Using the values of * end B (derived from A) corresponding to a 
particular value of Cumulative iTwlijct Research Expenditures, deter- 
mine from Figure lift (lie value of Process Research E-Xpenddures 

Repeating steps (J7)-(20) for the various values of Ciunulative Product 
Research Expenditures yields one curve for a iqiccified Net Return on Gross 
Value of Plant as shown in Figure IJe. 

Repeating steps (J)-(20) for various values of the Net Return on Gross 
Value ot Plant yields a family of curves such as those shown in Figure lie. 
Drch curve in the family dcscril/e* (he alternate eoinWnatioas of required 
levels of Cumulativo Prwluet and Process Re<^aTch Expenditures. 

It will be npp.arcnt to many that these inatnictions are quite similar 
lo many Bpccificatioiia o( cost-accotinling and data-processing systems 
(no matter how superficial might have l»«n (ho background renson- 
ing) and computer programs. 

Such instructions should bo pretested on membera of the population 
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who will U 5 e them in order to assure their completeness and lack of 
ambiguity. The feasibility of the overall implementation plan should 
also be checked with those who will cany it out. This is best ac- 
complished by inrohdng the operators in the preparation of the plan. 
We shall consider this aspect of the organization of the research ef- 
fort in more detail below. 


ORGANIZATION OF TIIE RESEARCH EFFORT 

Organization of the research effort involves all the participants in 
the problem: the decision makers, those who are affected by the ac- 
tions recommended by the research, and the researchers themselves. 
Each of these will be discussed in turn. 

The nature of the organizational problem obviously depends on the 
size of the research effort: the number of researchers involved and the 
duration of the research effort Since our concern in this discussion 
is with the relationship between researchers and other participants in 
the problem, it is most useful to have in mind a relatively small re- 
search effort in which each member of the research team is likely to 
have some direct contact with the other participants. Such a research 
effort will usually involve from one to eight or nine scientists. 

Even In large research efforts some subgroup of the researchers will 
have access to the other participants, and hence the following dis- 
cussion can, with minor modifications, be made applicable to such 
sltuatlcms. 

The Decision Makers 

A minute’s involvement of the decision maker (and those who cany 
out the decision) in the research before results are obtained is worth 
at least an hour of "selling” the results after they are obtained. 

There is a strong tendency among researchers to withdraw as much 
as possible to the isolation of their offices and laboratories to work 
out a solution. Such a procedure fails to attain the kind of involve- 
ment of the decision maker that is required for a high probability of 
acceptance and successful implementation. 

There are dber imporfant reasoos for Ibe iiirelrczzpcai is rerearrh 
of decision makers and those who will carry out the decisions. Con- 
tinuous review of the research by them ser\x5 as a safeguard against 
errors of omission or oversimplification. There is nothing that can be 
as frustrating to a researcher as a statement at the end of the proj- 
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ect, “T1)D research is fine and very impressive, but unfortunately you 
forgot to take *so and so' into recount and 'so and so’ is very inipor- 
(ant. Therefore, we can’t use your results.” 

I have found it very dcsirabie to /ormaiJze tljc participation of tJjo 
decision makers by organizing them into a Research Advisory Com- 
mittee and meeting with this committee as frequently as is productive, 
but no less frequently than every other month, to report on progress 
and plans. Informal meetings with individual decision makers arc 
held as required in the intenm. This facilitates a gradual absorption 
of research findings and promotes understanding. It also produces a 
sense of participation and involvement tliat usually removes the need 
to "sell" results. It also frequently leads to implementation of par- 
tial results before the research is completed. 

Although the amount of time required of decision makers during 
the course of research is generally small, it is not always easy to ol)- 
taln, particularly from executives who arc almost always verj* bus>’ 
men. In this coBnectioa there is an important principle which, if fol- 
lowed, can avoid considerable frustration for the rcsesreber; ntier 
perform research for a decinon maker for vhich he doet not pay. 
People ia responsible positions tend to be suspicious of the motives 
of anyone who tries to give them eomethiog for nothing. In general 
the amount and the quality of attention they give to an activity are 
proportional to the cost of that activity to them. An Individual or 
group that Is not willing to pay for research into a problem is not 
likely to give serious consideration to results obtained from a study 
of that problem. 

I have had experience working on successive problems for the same 
decision makers, first at a cost to them and subsequently with the 
research being paid for by others. Even under these conditions the 
amount of participation and cooperation provided in the second etu<ly 
was very much reduced. 

Motives of sponsors 

Not all persons who sponsor research into a problem are primarily 
motivated by an interest in obtaining a solution to the problem. The 
research may be used as a device to satfafy some personal motive in- 
volving status or prestige. Research, for some, a like an article of 
clothing worn for the impression which it makes on others rather 
than for the function it performs for the wearer. Where this is the 
case the researcher roust try mther to avoid involvement or to direct 
his ^orta so that the results ai« used os they ought to be, despite 
the sponsor. 
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In many instances research is solicitcU for Uic purpose of “prov* 
ing” the validity of a conclusion already reached by the decision 
maker. If the decision maker respects the integrity and standards of 
the researcher, he is not likely to disclose tfiis motivation until re- 
sults appear whicli are at variance with his preconceived notion. For 
this reason it is important that the researcher probe early the motives 
of the decision maker as thoroughly as possible and, if necessary, ob- 
tain an e.\p)icjt assertion to the effect that imposition of the precon- 
ceived notion will not occur during the research. 

In some cases even a wclUmtcnlioncd decision maker will obstruct 
progress of the research l>ccause of feelings of insecurity whicli ilcrivc 
from his relationship with others (usually above him) in the organi- 
zation. This kind of obstacle can frequently be removed once it is 
understood, and the researcher may be uniquely equipped to remove 
it. Understanding the nature of the insecurity, liowever, may require 
the special skills of the psj'chologist or sociologist. 

Guarding against reorganization 

Organizations are subject to almost continuous reorganization. 
Many research projects and tlieir results have boon washed down a 
drain because the person iidtiatmg the research is moved to another 
fjosition and his replacement Is either not interested in the Inherited 
research or is so occupied with his new responsibilities that he cannot 
take time to participate and inform bimself about the research. For 
this reason it is extremely desirable to involve as many persons in the 
responsible organizational unit as possible. For example, if the work 
is initialed by the director of a unit, his assistant should also be in- 
volved. Then if the director is replaced by his assistant, no hiatus 
occurs. If he is replaced by someone else and the assistant remains, 
lie can assume a major share of the task of bringing the new director 
up to date. 

Unselling 

The tone of the discussion up to this point implies that the decision 
maker is usually one who must be "sold.” Although this is frequently 
true, it is by no means always the case. In some situations the de- 
cision maker may, because of pressures being applied to him, be anx- 
ious to grasp any straw the researcher holds out in the wind. The 
task here may be to “imselJ” the decMoo makej-, to convince him 
that preliminary results are not substantiated enough to take action 
on. In such cases the decision maker is likely to transmit the pres- 
sure applied to him to the researchers. The researchers may be able 
to relieve the original source of pressure by e.Yp]aiDjDg why time is 
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required to find an adpfiuaie solution to the proijJcni and wJiy preHmt- 
tiary results are not to be trusted. 

In some cases "unseiling” toll be required because the decision 
makers are overcnlJiusiastic even though they ore under no pressure. 
The same kind of irrational enthusiasm vhich is displayed by many for 
electronic computers la sometimes <lisplaycd toward recently devel- 
oped and advanced scientific techniques. I have seen decision makers 
who were more interested in using the output of a scientific technique 
for its prestige \'alue (e g., linear |>rogramming) than in solving their 
problems. Tliia kind of attitude toward simulation and gaming, for 
example, is becoming increasingly prevalent among military and io- 
dustrial executives. 

Tlie possibility of the various kimis of difliciiltics wc have been 
considering empiiasizcs the need for effective communication between 
researchers and decision ninkers. It is necessary’ for each to learn a 
great deal about the otlier’s profession if effective use is to !« made 
of the results of scientific research. 

Those affected by deefrions 

Another common cause for Ibe failure of rcsiilu to be implemented 
is conUlned In a statement by decision makers sucli ns the following; 
"The results are fine and I'm sure they would help us a great deal, 
but wc do not have the kind of people who can enrry out the recom- 
mendations, " or "The people we have will not change their hchai-ior 
in the way required by (he solution." The safeguard against such 
remarks is to involve in Ibe researrb those who will carry cut tlic 
recommendations and to give tJicm a part in devcioplng the plan for 
implementation so that the decision maker can be confronte<t simul- 
taneously with research n^ults and a plan for their implementation 
develop^ (at least in part) by those who will carry it out. This also 
prevents the rcsearcli from being looked upon by the ''operators" a- 
a threat to tficir security, or ns a critidsm of their effecfiveBe<y!. 
Unthcr it is likely to be conridennl as an effort nt scir-improvemenl- 
1 have seen many situations in which cooperation of tlie operators 
was considered to ho unattainable by the decision maker but was 
obtained by the researchers as a result of their contact with the oper- 
ators. Let me briefly recount several such instances, because they 
may contain some oseftif guides to tfw o/gan/zaf/on awf conduct of 
research. 

In a study performed for one of the larger romincrcinl airlines it 
was found that by changing sclieduling procedures ad<lilional etev ard- 
c.'scs would not be required even thougti more flights were to t-e added. 
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A consequence of the proposed scheduling procedure, however, was an 
increase in the average number ol flying hours per stewardess per 
month. The increased average was within the limits set bj’ the union 
contract, but the personnel manager insisted that a large number of 
stewardesses w-ould quit if their flying was increased by the pro- 
jected amoimt. He agreed, however, to let the researchers look into 
the problem. 

First the researchers performed an analysis of available data which 
showed (a) that a significant percentage of the stewardesses were 
already fijung more than the projected average number of fi>’ing hours, 
and (b) that the senior stewardesses who bad first choice of assign- 
ments had no inclination to select those with fewer flying hours. 
Further analysis showed that what the atewardesses did prefer in 
assignments was as few stopovers away from their home base as pos- 
sible and regular days off. Discussion with stewardesses and their 
representatives confirmed (he results of the analysis. The scheduling 
procedures were then modified to pve them these desirable characteris- 
tics with virtually no loss in efficient}’. The final procedure was then 
acceptable to both management and the stewardesses. 

The second case involved sea transport For oany years the man- 
agers of steamship lines refused to consider the possibility of auto- 
mating cargo handling because of wbat they claimed to be a complete 
unwillingness os the part of longshoremen to cooperate. The director 
of a research group in a west coast steamship company persisted in 
his efforts to bring about automation and finally received permission 
to set up a joint study with Ihc research group in the longshoremen’s 
union. Thdr combined effort was directed toward finding ways in 
which the benefits of automation could be equitably divided among 
employers and employees. Such ways were found and led to the 
recent precedent-setting agreement between the companies and long- 
shoremen on the west coast. 

In a study directed toward deterroining how much time salesmen 
should spend with customers it was necessary to obtain accurate data 
on how the salesmen were dividing their time between planning visits, 
traveling, waiting to see customers, talking to customers, and so on 
Sales managers were sure that the salesmen would be unwilling to keep 
such records, since they would be interpreted as spying on them 
Again the researchers arranged for a series of meetings with salesmen 
with no managers present The reason for and the nature of the 
study were explained to them, but their reaction was as hostile and 
suspicious as predicted. At a critical point in the discussion one of 
Uie salesmen asked whether, il the proposed data were collected, it 
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would be possible to deWnninc the actual coat of selling each typo 
of product Jiandled and whether tills could be used, in turn, to evnl- 
unto current company pricing practices. At the mention of this pros- 
pect the salesmen became very inlercsteil. Their discussion revealed 
that they believed that they were losing business to (heir competi- 
tors because of improper pricing. In n short time they agreed to 
kceji the desircil records if the resenrehers would use them to examine 
the pricing practices, TTjo researchers obtained management agree- 
ment, and the data were collected as desired and were of even belter 
tpiality than the rescarclicrs h.ad expected. 

As indicated earlier, Uiesc and eindlar etperienees have a great deal 
tn teach us about how to obtain cooperation between apparently hos- 
tile parties, tfnfortunately science has been used more extensively 
to develop strategics of conflict and competition than to study strivtc- 
gies of cooperation. The necessary condition of cooperation dis- 
played In the examples just given is simple to state, but not easy to 
accomplish in practice: cooperation between parties whose interesU 
are in conflict can be obtained only by flnding a set of overriding coin- 
moa iateresti which can be rsore e/TecUvcly sen’ed by cooperation. 
RccDgnttion of these important common interests is seldom the result 
of insight or intuition, tt is tlie product of a thorough knowledge of 
tlic situation produced by sound rescarcli methods and techniques and 
by cITective communication between the parties involved, usually 
through a channel consisting of persona who arc not a party to the 
original conflict. Researchers often arc uniquely equipped to provide 
the channel, as well as the knowledge required to resoh-e such con- 
flicts, particularly researchers ii'bo are not employed by either of the 
parties involved. 

In my research e.xpcrienec I have never encountered a conflict be- 
tween decision makers and operators which could not be resolved by 
researchers who had unlimitisl access to the hostile parties. Since 
the decision maker usually sponsors the research, he must be "sold" 
on the desirability of this unlimited access to operators. Such selling 
requires that the mystery assodated with "negotiation” be dispelled. 
Unfortunately many managers feel that their security is tlireatcned by 
dispelling this mystery. This, however, is but another conflict that 
must be resolved, and the same approach is applicable to it. 

The Rescarcli Team 

For reasons which will be discoased below and In the last chapter, 
research teams— rather than individuals working alone— are being 
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used iacrcasingfy for tlic solution of problems, ilost of pure science 
is still being conducted by individuals worlang within a scientific 
discipline, but in tlie area of appHcatton teams of mixed scientific 
disciplines arc more and more cointnon. 

Tlic use of research teams — particularly ones involving several sci- 
entific disciplines — creates certain orgaoiratiooa] problems iiluch, if 
not solved satisfactorily, can have a detrimental effect on the research. 
A team of researchers may be either less than or more than the sum 
of its parts. It will be less than the sum of its parts if friction ab- 
sorbs a great deal of the energy expended and if the parts simply 
augment each other's weaknesses. It is important, tljerefore, to com- 
pose a team in such a way as to compliment one man’s weakness with 
anoUjcr man’s strenglL 

Scientists' personalities in general can be placed into two major 
categories which correspond to two major philosophies of science: ra- 
tionalists and empiricists. The rationalist is a fonnalizer who likes 
to deal with the abstract and hence proceeds to a model or theory as 
rapidly as he can. Ho resorts to data as little as possible, and some- 
times not even this much. In the extreme, iie is one who likes to find 
problems which fit his techmques. 

The antifonnallst or empiricist, on the other hand, insists on know- 
ing aU the facts before /oraolatiog the problem or an approach to 
it. Since the pit that contains data is usually bottomless, he generally 
feels that he nei'cr knows quite enough to formulate the probleoj. 
He tends to deal with the specific and concrete, whereas the rational- 
ist tends to deal with the general and abstract.* 

Few scientists are as extreme as the sketches indicate, but tenden- 
cies in these directions are usually easily discemable. By combining 
these ^pes of scientific personality the team U given a more complete 
personality than any of its parts possesses. The rationalist and em- 
piricist tend to supplement each other. The rationalist presses the 
empiricist to organize his data into a precise formulation of problem 
and theory. The empiricist forces the rationalist to relate his abstract 
generalizations to the problem at band. 

Another dichotomy whirii affects team performance involves the 
degree of familiarity with the area in which the problem exists. A 
person tborou^ly familiar with a problem area is mually restricted 
by many implicit assumptions conceniiDg tlie area of which he is not 

• For sn iUaminsting discusion ol the difference between the “formalist” ami 
the “realist," see Mina.a (1936). 
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aware. He tends to be unable to «e alternative solutions or to discard 
suggestioos 0/ new flltemath-es an the grounds that they have been 
trieil before and failed. A person completely unfamiliar with the area 
is JikeJy to tiiinfc of all kinds of alternatives, but lie is not likely to 
be in a very good position to eA'aluate them realistically. By com- 
bining into 0 team ^ili those who are familhar and those who are un- 
familiar with the problem area, assurance is gained that a wide range 
of alternatives will be considered and tliat they will be cv-amined in 
light of the facts of the case. 

The interdisciplinary team 

There is a natural Inclination on the part of those who have a prob- 
lem to classify it by scientific or technological discipline and therefore 
set up a research team consisting cxcfusivciy of persons in that disci- 
pline. A strong counlertendcncy to develop interdisciplinanans and 
use intcrdisciplinari' teams is presently evident. This recent surge of 
interdisciplinary activity has Us roots in the hUtory of science and the 
nature of problems being c.xposed to science 

Contemporary scientists and consumers of science tend to overlook 
an important and interesting fact about the development of science. 
Until about the middle of the ninctcentli century what we today refer 
to as science was conducted under Ibc name of "natural philosophy.’' 
The inventory of knowledge produced by this jiursuit became too full 
for one man to store in bis mind. Specialisation was inevitable, and it 
came about through the separation of science from philosophy, the 
organization of scientific societies, and the formation of departments 
of science in universities. 

T\'e also have a tendency to think of the division of ecience into 
its various brarcJies as corresponding to a natural classification of 
phenomena. The dirtsion of science into disciplines was accomplished 
bj' man, not by nature. The disciplines cannot be individuated by a 
tinifjue set of objects or events which constitute their subject matter, 
but they are distinguished by the aspects of phenonjen.s to which they 
direct their attention. Many phenomena may be studied fruitfully by 
each branch of science, but each will concern itself nilh different 
aspects of the phenomena. For emmple, a communicative act may 
have physical, chemical, biological, psyclioiogical, eociohsica}. and 
economic aspects. 

In the last hundred years it has been necessary to continue to 
divide onr study of plicoomcna toto more ond more simemtaed do- 
mains so that progress in deptJi could be maintained. e egnn 0 
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Teach a point, in the second quarter of this centurjv where the prob- 
lems most pressing to mankind could no longer be fruitfully treated 
by specialists. A synthesis of scienUfic disciplines was required. The 
effort to create scientific eencralists, a much-discussed znorement of 
the thirties, was directed along these lines; but it was doomed to 
failure because it acquired breadth at the cost of depth. Research by 
interdisciplinary teams was the onty feasible alternative. 

The second quarter of our century saw numerous interdisciplinaiy 
team efforts. One such movement, philosophical in orientation, was 
the movement for a unified science. It produced the International 
Encyclopedia of Unified Science. An Institute of Experimental 
Method was organiied at the University of Pennsylvania in the early 
forties and conducted interdisciplinary research on a variety of prob- 
lems involving large systems. (See, for example, Churchman et al. 
(1947).] These are but instances of a trend which Kenneth Boulding 
(1956, pp. 199-200) has described as follows: 

la recent years there has been a . . . development of great interest in the 
form of "milltisexual" interdiseiplines. CybemeUts, for instance, comes out 
of electrical engineering, neurophjtiology, ph^-rics, biology, with even a dash 
of economies. . . • 

On the more empirical and practical ride the interdisciplinary movement 
is reSeeted in the development of interdepartmental institutes of nmy kinds. 
. . . Even more important than these visible developments, perhaps, though 
harder to perceive and identify, b a groning dissatbfacUon in many [univer- 
ritf] departments, especially at the level of graduate study, with the exist- 
ing traditional theoretical luckgrounds for the empirical studies which form 
the major part of the output of Ph.D. theses. 

The increasing use of interdisciplinary research teams results from 
the growing recognition of the fact that a large number of problems 
cannot be solved effectively within the boundaries of any one disci- 
pline. In tbe operation of a system, for example, the aspects of the 
system which can be manipulated so as to improve its performance are 
likely to come from many different disciplines. The determination 
of which of these variables can most effectively be manipulated from 
a control point of view requires the j<anl analysis of experts in the 
areas in which these variables are normally treated. Such joint 
analysis permits the construction of models in which variables from 
different d’lsciplines interact. 

The interdisciplinary approach to problems does not consist of n 
sjTithesis of research results obtained by independent tnfradisciplinarj’ 
studies of the 5.ame phenomena. That is, it is not accomplished by 
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first setting up disciplinary teams to study the same problem and then 
attempting to relate the results that arc obtained in this way. Inter- 
disciplinaiy researcli docs not begin at the end of disciplinary mqumes 
but at the beginning of Uic research. , 

In practice, of course, it is seldom possible to have a wide variety 
of disciplines represented on a single research team. It is possible, 
liowevcr, to comult with rcpMcnUlic-cs ot « tcido vanety ol 
nnd to use people on the team who ate aware ol the content and mett 
oda ol other disciplines. It is no accident that so many 
scientists and engineers have multidBcpIinary cdneation.l back 
grounds. 

Norefey^SM works well in an iaterdiseiplinary environmenk 
In my e;pT,ieneo”the principal "1.0^';';™!“? 

collaboration is seienliBc snobbery. Such anobte^ LS mSS 
It is rooted in an assnmed hierarchy of ” “"“X 

SJ'vrs" 

respect, and respect must >>' b“ .J„dice is^mest Stmly tooted in 

As in most areas ol human region, ptci^^ 

ground well lerliliied y. Hj/progtess has been re- 

laek the dignity ol age. Unbl ^ matter They are subject 
tarded by denial ol access pjrtmnlarly the polilieian. 

lre;TwS. the most perpleaiag subject o. study 

>tS‘:ryjhr. great d., >» " 

with respect to content, but also \n ' . j ^ behavioral scien- 
niques. To mention but concepts and 

lists have yet to learn and Zxt,. isonds of physical scien- 

techniques which have been p ^ j„ ijjn, and esploit the 

lists. On the other hand, the latter y pmjuetive m 

statistical concepts and , ,*„Sst,. 

the hands ot the biological and L „irfy on scientinc 

Now that most ot the ^ 0 : which remain arc 

progress have been removed, the pnneipa 
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those imposed by science itself. Ironically, society is now applying 
pressure to remove these obstacles and will undoubtedly succeed in 
time. 


SCIENTIFIC DEMEANOR 

There is an old saying to the effect tliat a man is judged by what 
he stands for. There is considerable ambiguity, however, in the mean- 
ing of “stands for.” On the one hand, it refers to what a man will 
tolerate, and on the other to what he will not tolerate. The principles 
of a man are evaluated ns much by what he refuses to do as by what 
he docs. These negath'c acts clearly define his principles in the 
eyes of the spectator. The amount of compromise of scientific prin- 
ciples which a scientist is expected to make reflects the amount he 
has demonstrated a willingness lo make. 

A scientist must be prepared lo refuse to perform work under condi- 
tions which do not permit him lo meet what he considers to be mini- 
mal acceptable scientifie standards. To the extent that ho demands 
satisfaction of these conditions he will command the respect of the 
consumers of research. 

The ability to stand on principle and to survive depends, of course, 
on the scientist's ability to perform effectively under the conditions 
he specifles. Freedom of research action is related to the ability of 
the researcher, Hence the scientist’s best strategy is always to make 
himself as effective a researcher os possible. 

The most powerful weapon a good scientist can have against abuse 
is the implicit or explicit threat to quit. Such a threat should not be 
made unless the conditions necessary for his remaining are made ex- 
plicit and unless he is fully prepared to leave if they are not met. 
Bluffing is not an effective research strategy. 

In effect a scientist should never perfonn research which he would 
not be willing to publish in a professional journal at its completion. 

A researcher’s performance depends on what he knows. Knowledge 
decays and is replaced. Therefore, keeping up with the developments 
in a field is an essential part of a scientist’s activity. To do so 
involves extensive reading, convereaUon, attendance at professional 
meetings, and so on. Such activities should be a planned part of a 
scientist’s job. Nonscientists who employ scientists are frequently 
unaware of the importance of such activities. It is necessary in such 
eases to impress them with the importance of “keeping up.” This 
education ol research consumers is the researcher’s responsibility. 
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Dissemination of Research Findings 

Such difsenimation is an irojwrtant aspect of science, pure or ap- 
plied. Militar>’ and industrial security has become an increasingly 
serious harrier to effective scientific communication. The scientist 
c.nn too easily place responsibility for ouch a slate of affairs on others, 
hut the responsibility is ultimately his. Security regulations can also 
easily become a screen behind which cither poor or mediocre research 
be bidden and/or behind .hieh important reseatel. lind.nse a e 
mlidttid Jroin olhera who can me them effectively. Scienteti m 
inercasinsly reserve the right to publish at least part of tl.eir 

""itright be foolish to deoy that there is any “>7“' 
security restrictions on seienee-hol an 77" 

«,e taUng this “foolish” positioo. nowever, 
ablencsa of eeeurity measure, some mioua ‘>>”"7 '' 
to how much of research they should appy “■ J 
product, produced by rcse.reh, or a eo«pel,uve 

developed,migl,tbe resemd 

fend placing "f <uSn“ beteL mcli.od (and lechaiciuci 
wrap” except where the dmeren^ oev ...uallv advances re- 

and eonteot is hard to define? Cock - should be pub- 

search inetliodology, and this mueh, a. a mmunnm, should P 

'1.1; the responsibility 01 ilierrtc^^^^^ 

to the distinction between the co j, 

™tL°r°o rbright imlee-n .lenied to the competent seienUst who 

were working in complete isolaUou. r.^ery 

to add to the public pool of itself. Scientific 

Publication, of course, trivial and redundant material. 
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COMMUNICATION 

Many books have been written on how to communicate effectively. 
The intention here is not to summariic tins large body of literature. 
Rather we will consider a few — but, I hope — fundamental observ'a- 
tions and principles that apply to the more formalized reporting of 
research. Some of these remarks will have relevance to day-to-day 
informal communications between the researcher and other partici- 
pants in the problem. 

It seems to me that individuals arc less capable of objective ap- 
praisal of their communicative efforts than of almost any other aspect 
of their behavior. Most individuals, and particularly those with 
higher education, feel that they do a good — if not excellent — ^job of 
speaking and writing. Most don't. Because people are so easily of- 
fended when criticized in this area, most of us hesitate to tell them 
what we really think of their communicative efforts. By doing so we 
strengthen their illusions. Consequently few adults improve in this 
respect with practice. 

Improvement is possible, of course, but it must begin with an ob- 
jective appraisal of deficiencies. Such an appraisal can usually be 
obtained from others. Sincere requests for criticism, particularly 
before final presentation, are usually rewarded by disturbingly frank 
comments. 

Presentations of the results of research (orally or in writing) should 
be made by that member of the research team who is the most ef- 
fective commimicator, whether or not he directs the effort. Conven- 
tion, however, dictates that the senior member of the team do so. If 
he is not an effective communicator, his vanity may be satisfied by 
having him make brief opening and closing remarks, but someone else 
should make the main presentation. 

Empathy with the Audience 

All the advice and instruction on communication would be unneces- 
sary if each speaker or writer could hear or see his output as his audi- 
ence does. Empathy with the audience is the most valuable asset a 
communicator can have; that is, senritivity to their response to him. 
Too many of us write and speak to an imaginary audience consisting 
only of persons identical to ourselves in every respect. To take a 
common example, how often have you been imable to decipher slides, 
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charts, or writing on n blackboanl used by a lecturer? Even the 
Blightest amount of empathy m a speaker would lead him to make sure 
his visual aids could be seen by all present. But let us consider a 
more eerious manifestation of Uiia insensitivity to an audience. 

The great adv.antngc of presenting research results orally is 
feedback is possible. Questions can be asked and answercil. But 
unfortunately the fiuestions asketl and those answercil are fluently 
not tho tailie. Many a speate-a imaginatwn l. a Lamer “hl'l' P''’ 
vents n question Irotn Leine I, card as it is asked Tliere ore too s.mplo 
precautions llial can lie taken to avoid or rcctity siieh roismkrpreto- 
lion. First, the speaker can restate the question and ask the qne^ 
tioner i( he has done so eomatlly. H n«l, l« y 

inent until it is aeeeplahle to the questioner. 
can ask the questioner, after attempUng to answer O™ , 

has done so s^tsfaelorily The-c pq.cl.ces 
control, but they can train one to listen to others a 

Organization of Iteports 

Aristotle once obrerved that 

ginning, a middle, and nn end J" should be provided, and 

lion of what will be presented in ‘'j* “ p^scnlcd sliould also 

in the last section a to present technica 

bo provided. Too many speakers and wntere 

material in the style of 0. appew beginning, 

technical writing surprises shou , , j j ,jg quickly sb 

The listener or reader -"‘J *» Ih. P.int a, quickly 
possible, and then f ( „l i„U) details in discussion or 

nsposBible. Til'! aHdienec prefOT B The importance 

appendices, not in the roam body ,,nn„t he overemphasized, 
of direetnes., brevity', and oft iotporUnt. 

This is not to say that ^bc d search project shouid yield a 
To tlic conlraiy, they arc. ^ aunlicate the study in essential 
document which makes research lies in its ability to be 

detail, since the ultimate test , progress depends on the 

duplicated. Certainly the pros 

ability to study past work m great detaiL 


Tlio 


ho Language of Reports 

A, has been indicated, a "'rSn^ferifn ta "‘to 
:aUou is that the language and mode 
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audience, not of the speaker if liis differ from theirs. Researchers arc 
inclined to defend obscurity on the grounds that the ideas invoircil 
arc complex, but it is more likely to reflect an unwillingness to take 
the time to translate ideas into nontechnical Inngiiagc. The researcher 
should not consider this transIaUon problem ns a chore, but as a chal- 
lenge. He should appreciate tlie fact that in making such a translation 
he subjects his work to the most severe critical evaluation of which he 
is capable. It is very’ easy to conceal glib assumptions from oneself 
and others by the use of symbols and technical jargon. As one ap- 
proaches expression in Basic Englisli, however, self-deception and 
deception of others become increasingly diilieuU. Simplicity of expres- 
sion, like brevity, is the result of extended distillation and cv'aluation 
of ideas; it takes a long time and much effort to attain. 

It is sometimes helpful for the researcher to imagine he is gliost- 
•writing a report to be delivered by one of his audience. This may help 
him to get into the appropriate frame of mind for producing a report 
that is suitable for his audience. 

Slmulatioa as Display 

One of the major difficulties in eommunication lies in explaining a 
model or solution which contains large numbers of variables in com- 
plex relationships •nith one another. Any translation of the symbolic 
models into ordinary language seems lengthy and too complex. In 
such cases the use of analogues that can be seen and manipulated can 
come to our aid. Simulation, in short, is a very effective communica- 
tion device. It can be used not only to describe, but also to explain 
a model and solution, and frequently it will succeed in justifying a 
result in the eyes of the decision maker where other approaches com- 
pletely fail. 

SUMMARY 

It has been argued that the researcher is responsible for producing 
recommendations in applied research and that he must also assume 
responsibility for and leadership in cany’ing out these recommenda- 
tions if they are accepted This requires detailed planning of the im- 
plementation, including programming the action to be taken fay each 
individual involved. 

In order to assure the soundness of results and their acceptability. 
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tliC researcher shouM involve the decHion makers and those vlio will 
be alecled by the decision in the research process, Tl)e more respon- 
sibility they can he made to (eel for the research outcome, the more 
likely the acceptance and implementation are. 

Where research involves a group effort, the team should be put to- 
gether with thought given to the miitiire of personalities and tjTcs ot 
training. Increasing use should he made of interdisephnao- teams 
because the complexity ot moat modem problems prevents their mc- 
eesalul attack by any one dLaeipline. The prmepa obstacle to effec- 
tive inlerdisciplmary coopetaUon is ecicntific snobbciy, which in the 
main is bom of ignorance. Mutual respect among disciplines must 
develop out ot an understanding by each disciplinarian of what other 
scientific disciplines can contribute to research. 

It the scientist is to be ot ma.vimum use to those who 

Titat r'eSMuS m"Stmr.e .5 srim.r'undsrd. arc necessary 

ft "ttsrt^ 

^thenooscieotiststhescien^ 

h 8 audience and make his prcscnwuw .hould re- 

These proseutation. should ^d “'rdS',"; d”or ap- 

veal their content at the outset ana bitten, 

pendices, depeoding oo 'srimee" involves nooseieoUsts in 

An ineveasmg amount ot P”” ,,, ^earcli much of what has 

the role of sponsors or subjects. , ,^arch U applicable. For 

been discussed in 5'’® from the layman, of 

the laboratory scientist woiki g .jj ^ problem of corn- 

course, most of this chapter bas no b/,pproaehed with 

munication among "pure" 

the same point of view as has been presented here. 
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THE IDEALS OF 
SCIENCE AND SOCIETY: 

AN EPILOGUE 


TIIE TOEALS OF MAN 

A problem never exists in isolfttion; ^ problem thfit a 

Icma in space and time. Tlic more o • finding ft 

scientist can comprehend, the groa .Snte annlics to ecicnee itself; 
truly adequate solution. This same 

significance of science is understood y 

into tho bniador ichrmt with tho quotioa: what 

Bcinatiat. oonttaally “"'^"‘“^j'ftcir.nawar, it provito M. 
is the social role of science? jcscareli problems arc eelcetcd 

nttitudinal background ngaiM .on«iderabtc influence on wlial 

and research is conducted; Jt nas » 

they do and bow llicy do it intention is to cmplmsirc 

In turning attention to answer it self-consciously 

the importance of the scientist a efforts to 

to both science and society- , , JiiJcjophcri have sought to 

Since the dawn ol recorded IhwM P obicetivee to 

idcnlliy the idenU ol .pprerimaliooi without tte 

which mnnkind seek. ! .„iV„t philo«phers idcntiBcd 

cspcclnllon ot eomplcle “‘'T"™': Modern phitoophrrr 

thL such ideels: <ru«. t ""“"Ik nbundonce) 

hove idenllOrd » lourtb: P'“'P ,( j, .„.u.cr to deSn. .nd 

It is one thine to P™e Ihce^-' • 
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justify them. Before attempting this diflicult task let us move back 
ti step. 

Moralists and anthropologista have long argued that there is no 
single objective that all men have souglit. Put another way, it seems 
that, if all who have lived and do live were granted one wish, there 
would not be universal agreement in that which w'as wished. Although 
wise men may find no universal wish, children do. When thej* are 
confronted with the familiar story involving the granting of tlie three 
wishes, they frequently observe that if they had the opportunity they 
would need only one wish. The wish that they would make, of course, 
is that all tlicir wishes would come true. 

In a considerably more sophisticated way, E. A. Singer (1923, p. 
109) obsen-ed that accompanying every desire is the implicit desire 
for the power of attainment. 

Now suppose there were something >ou would have to have whether >o'i 
wanted to bisect « line, or break a safe, or do anything eJ'O in the norid; 
then, whatever you luppened to desire, you would have to desire that eome- 
thing ... the condition of attaining any end in the world is such control of 
Che irorid's machinery as shall give you power to get what you want. 

The desire for the abilUy to attain objectives is universal. If this 
ideal were satisfied, all others would be. Now when wo analyse tviiat 
is involved in the pursuit of such an ideal we can find the relevance 
of the Ideals of plenty, truth, goodness, and beauty, although the 
connection is by no means obvious. 

Men pursue objectives (ends or goals) by various means (courses 
of action) which incorporate a variety of conceptual and piiysic^l 
instruments. The capability for obtaining any objective presupposes 
the attainment of four ideals: 

(1) The scientific ideal of perfect knoidedge (i.c., complete attain- 
inent of truth) ; the ability of every individual to develop and select 
instruments and courses of action which are perfectly efficient for 
the attainment of any end. 

(2) The polilico-econontic ideal of plenty or abundance: the avail- 
ability to every individual of courses of action and the instruments 
necessary for them which are perfectly efficient for the attainment of 
any end. 

(3) T?to clkical-moral ideal of ffoad/MSs: (a) the absence of «k?- 
trary and contradictory objectives vrithin each individual (i.e., peace 
of mind), since no state of plenty or knowledge can make the attain- 
ment of such objectives possible; (5) the absence of conflicting objec- 
tives among people (i.e., peace on earth, good will toward wan), since 
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only in Ibc nl.«cnce of such conflict cMi «i-fry individual attain his 
olOcctivcP. . . .1 

These three ideals would be ^'ftcuou^Iy allnined if no one wanted 
nnytinns. TIic ..nivcrsnl tor U.e .Wily lo »tW'' 
iL-clt |msil|.p<nC 3 Ih.n llion- ore Jaircd ends (l c., irnlumilcJ objee- 
lives) and Hint Ihcy ronlimnwsly Miami, llclicr, Ibr In.sl ideal. 

(4) The aeslliello ideal «/ li«o»<»l 
tial's cnvironmciil ol stimuli mliirli inJjiirc Inm 1" 'aie' uspi'aUb”' 
to enlarge the scope nnd meaning of Ins experience. 

TImt pursuit ol llino ideals I, a. objMlivc raalily and -P™;'™”' 
..lEiiilie.ucc is apparcnl trom on essiuinslion ol llie ira> ™ 
stactured and the ivay they Hmetio" Uerv^d thf ™1 a 

lure In Icnn, ol licliaviorsl inslituUons .1 sliould be 
principal dlllcrcnccs belirem eocielies lie not so J 

hey pursue, but r.Uier in llie relalive iinporlance lliej 
ideL. Ihat’ls, in Ihe ivay they .lloe.le llieir . 

llieie ideals. There arc nl.sn dillerenees, ol 

soclctica consider to be the ] ll 3 pursuit may 

Consider first ll.c politico-ceonomie ideal of plent>. lla pursuit may 

be broken down as follows: 

Provision ol the In-lninieoU 

pr'oer ,SS”4Sre, t-s 

wliieh they nre ™l'"i„l|,Urinsin.lilution ol sM 

are the vanous Iransportation individuals ol the avsiiability ol 

(e) Creslion ol an sienesa j!,nnned by various inslilutions 

these instruments. Tliw function i f* j example, advcrtis- 

involvcd in the dissemination of ^ 

in,, tor obtaining lb. in- 

(tf) Provision of individuals wun i purchasing power 

stmmenta they desire; niay be accomplished by the 

for the instrumonU tlicmselvcf. , - other of his resources (e.g., 

individual through the sale of bw labor themselves 

lending money). ^pmeS Inr by such privsl. 

pitals. 
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(e) Protection of the avajiability of instruments once they arc ob- 
tained (1) against expropriation from witldn tbe social group, by 
police and judicial institutions, and (2) from forces outside Ibe 
social unit, by military institutions. 

The social institutions enumerated are all directed toward increas- 
ing the availability of instruments for the satisfaction of desires, not 
toward tlie knowledge of how to use llii'sc instruments effectively or 
how to develop more effective instruments. Tlicse Latter functions arc 
involved with the scientific pursuit of knowlctlge or tnith. 

The function of inslnictfon in the use of instruments, physical or 
conceptual, is performed in society by eilucalional institutions. Tlie 
development of more efficient means and instruments for the attain- 
ment of objectives is the function of applied science, and the pro- 
vision of the knowledge necessary for performing this function is the 
task of pure scienee. 

In order for an individual to attain all bis objectives there are botli 
personal and social requirements that roust be satisfied by the objec- 
tives he seeks. On the personal side his objectives must be consistent 
(that is, logically compatible) if he is to attain all of themj he can- 
not want contradictory things. Objectives which are practically in- 
compatible arc not the same thing as objectives wliich are logically 
incompatible. Practical incompatibility (e.g., we cannot afford to 
buy tno things we want, only one) arises out of scarcity of resources 
and would be removed in a "slate of plenty." Objectives which arc 
logically incompatible cajiaot be simultaneously attained under any 
conditions; the attainment of one implies the failure to attain the 
other. 

The simultaneous pursuit of contradictory objectives can lead to 
mental and emotional illness. Hence, the institutions of society de- 
voted to mental health are largely occupied with the removal and 
prevention of internal conflict, the promotion of peace of mind. Pro- 
motion of such a slate has long been an important objective of many 
religious institutions and tliw ethical systems. 

Conflict of objectives among individuals must be removed before 
each individual can attain whatever he desires. That is, every in- 
dividual's objectives must be compatible with everyone else’s. In 
the ideal, progress of any individual toward his objectives would imply 
progress for everyone else toward his objectives. This would be a 
state of peace, a completely cooperative society. Morality has had 
the task of moving society toward such a state. The promotion of 
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moralUv in this ssnss has b™ an objcrUsT o! many ethical and edn- 
cational institutions. _ . , />_:*„ nf 

Man’s concept of the ideal state hardly mro vcs a 
objectives which he would he able to sattsfy w.lh ™ 

.ouh! be much too static a utopia, ml another 
rccognired that there is a value in the slnvrng as ''f/' JY™. 

iuE. Therefore, his 'continuous striv- 

proachablc but unattainable state T1 P ^ ^ . 

ing must be energised in some nray. Man has found ttro wajs of clo g 

'°The oonlinuou. strivins lor an “t^man^^Ilol'e^m; 

tally and physically. In » »**>«“' /,t „e"lte°fnte. peri- 
new means and conceive new ends. recreate the 

odically for him to have a change ® j* Complex social institu- 
creator. This is the function of j requirements of con- 

lions exist in order to satisfy U.e recreational req 

tinuoua progressive striving. emotional, mental, and 

Recreation is a cathartic which aUows man^^ 
physical forces to be regenerated. “ as-yet-unatlained possi- 
and be stimulated to take tUro- J continually to progress to- 
bilities must be provided if ['« « he must bo inspired, 

ward that wliich he has not y function of religion, phil- 

Inspiration has long been . creates what might be thought 

osophy. and art. Through beauty « ^ pldlesephy paint 

of as the creative mood. Throu^ . 

a picture of what man and h correlative activities which as- 

Thus recreation and pursuit of ideals. . 

sure the dynamic properties of nia p desire 

^^Tiat we have done ^^^XTbility to aaUsfy their de- 

common to all men, the desire j. p^^cr over but power lo. 

eirtH. This objective, then, is . It can be ap- 

Complete and perfect ‘Attained. Pursuit of this ideal re- 
proached continuously but j,ondiUona which we have call^ 

Suircs progress toward ‘^UtuK We hm-c tried to make 
plenty, truth, the good. »"f^^’g^’^h5cal-moral, and aesthetic ideal 
these politico-economic, r^jjesting the social institutions deleted 
operationally meaningful by «« 
to their pursuit. s„.tltuUons interact strongly. 

The four ideals and m- . politico-economic ideal y 

o2llXot«W,J to tfe rn^l „„ duiribuu™ 

developing more „g protecting these against agg 

and services, and better f 
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Science also contributes to tiie pursnit of the ethical ideal and has 
been doing so increasingly in the last few decades. It has developed 
procedures for determining what people desire, what the relative im- 
portance to them of these desires is, and how conflicts within and be- 
tween individuals can be resolved. Science has contributed least to 
the pursuit of the aesthetic ideal It has tried to study the art forms 
which produce an aesthetic response and to gain understanding of the 
nature of this response. Nevertheless, a science of aesthetics is more 
a possibility than a reality. 

We can turn the coin and consider what might be called the politico- 
economic, ethical-moral, and aesthetic aspects of science. In doing 
so 1 believe we can cast light on some of the more important prob- 
lems now confronting science as a social institution. 


THE POUnCO-ECO.NONnC tDE.\L OF SCIENCE 

Tc proi’ufe every eeienlUt tcilh the physical and conceplwil 
instruments necessary for the eficicnl purrutl of knovhdge. 

Physical tnstnmieots 

Scientifle progress has been closely associated with the develop- 
ment of phyrical instruments; for example, microscopes, telescopes, 
microroetere, thermometers, nuclear reactors, and analogue and digital 
computers. The complexity and hence the cost of equipment required 
by scientists have been continuously increasing. This has tended to 
restrict research more and more to large academic, governmental, 
industrial, and research orgamzations. The isolated research worker 
who is tucked away in a small institution or garret has become a rarity 
in all the sciences. 

This centralization of research activities has tended to divide aca- 
demic institutions into those which concentrate on teaching and those 
which concentrate on learzusg (that is, tfiose in which research plays 
an important role). Such a separation can haw serious consequences 
to teaching, some of which have been recognized and are recehmg 
more and mare attention. For example, largely due to the National 
Sdtnte ttktre has Veen a •rirtaai rash o! summer iatfi 

other types of special) programs for faculty members of smaller col- 
leges and public schools at larger institutions so that they can keep 
up with recent scientifle devclopmeats. 

There has also been a concentration of industrial research and 
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hence innovntion in large companies. In the United Kingdom associ- 
ations of smaller conipanica have been fonneil to carry on rKcarch 
ttliicli could not he supported by any of tlie members alone. Only a 
relatively few such associations have appeared m llie United States. 
As a consequence, small businesses find it increasingly difficult to sur- 
vive here. , . _... 

To help rciocJy the Mailcmic problem, nisocations of 
on.l eollegee have been Iomie .1 to rise larce eommon research 
tor cample, the Aes.eiation ol Mid.-esl Umversibes, rr h.el, .a tal 
around the ta.ililies of the Atonri. Eneror Comm,.s.o«a ArEMoe 
National Laboratoriee. There ia no reason " 

universities nnd colleges cannot tom. such assoemtrons W “ 
tivcly develop common taeil.tiea tor research. 
inerised travel and ineonrenieaee tor those ‘ 

certainly preterable to being redoced to 
rather than acting as a participaat ia the scjen i c ' j ^ 

Th. observer ol these developmeati f'P 

in tho future n great deal more jn tlie past, 

search organiratioas will he requir^ th -jjjy .3 needed, there 
It cooperative association, do ,ad t 11^^^^^^^ 

will bo an cvcr-deepenmg The undesirability 

and fine arts, and between teachmg and tarmag r 
ol such a chasm is too obvious to require drseoss.on. 

Conceptual Instruments 

The availability of «?nS 

instruments, is more a nts__making their availability 

The distribution of conceptual m them— is the problem of 

known and providing r^problero which is consuming an 

scientific communication, im The volume of “ms- 

increasing amount of “L„n increasing at a much greater 

lerinl” produced by science h ^ .l>sorb it or even to obtain 

rate than the ability of „ -long these lines a number of 

access to it. In order to make successfully solved than they 

fundamental problems must be more successlu ly 

have been to date. ,i-vote himself exclusively to keeping 

Even if a scientist were to difficult is due in part 

up Avith his field, he could n ^ 

• For cmple. » b 'f ' .K" ''' "" 

rtadinB only journals, Resrarth G«>UP 

journal articles publthed lOperat.ons i«« 
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to two conditions. First, too much time is required to determine what 
material is relevant; hence more effective filtration of information 
is required. Second, the information whicii is relevant is too fre- 
quently packaged in such a way that too much time is required to 
“open it up.” There is too much useless redundancy in scientific 
literature and too many words which convey no information. The 
problem here is one of “data reduction” and “condensation.” 

The time required to locale and retrieve useful information has 
become excessive, and the problems involved in the physical storage 
and handling of information have become ovcra’helming. New instru- 
ments and means of retrieval arc badly needed. 

There is some hope in the fact that these problems arc receiving 
increasing amounts of attention from scientists themselves. The recent 
cstablislimcot of the OfBce of Scientific Information Service in the 
National Science Foundation and tlic more recent enlargement of its 
responsibilities evidence the growing social concern with the problem. 
ISco Proceeding* (195S) and Brownson (lOCO).] The problem be- 
came a matter of public discussion for the first time wiien the de- 
ficiencies in the eurrent eeientifie information system in the Unl(e<l 
Slates were blamed in part for the superiority of achievement of 
scientists in tlio USSR relative to the exploration of space. 

Progress In handling these communication problems is not going to 
come easily. It is clear that good solutions will require scientists and 
research of high qu.slity. In the past, research in this area has not 
nltraclcd our most cap-nWc scientists, so Ihnt iIjd large volume of 
verbal output contains %-cri' lUllc by way of operationally useful 
results. 

A prerequisite for the effective study of an activity, he it scientific 
communication or industrial production, is good record keeping. 
Wlicrcas the re.«careli Ecienlist is the first to conipkain about Ujc rcc* 
ortl keeping of others whom he may study, he is generally the hi't 
to oliscr>'c Uic inadequacy of the records he keeps on his own activity. 
Furthermore, he tends to feel (hat there h fometliing degrading afmiit 
keeping rcoor<}f. 

To a large extent the poor stale of scicnUfic pommunicnlion is due 
to very meagre knowledge wc have of how scientists tiy to use 
the literature and the roic that literature play* m their work. The 
scientist is no beUcr at HTlro«pcction on these matters than tlie indus- 
trial man.'iger is on liow be spends his time or uses the information 
supplied by his suboniinatc*. U is only by an objective accounting 
of what cine docs that one enn examine the efficiency with which it is 
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ment does not fall into that person’s hands. This principle applies to 
the scientist and his handling of the results of his researcii. 

Now this principle may not scan to apply when the instrument in- 
volved can be used by the rcciiMent to protect himself against unjusti- 
fied attack by others-, that is, when the instrument is more likely to 
help than to harm the recipient In this case, then, the critical deci- 
sion involves weighing the potential harm against (he potential help, 
and tliis decision must be made by the scientist. To be sure, an in- 
dividual scientist may not be able to predict the social consequences 
of releasing the results of his research, but other scientists may, and 
he is obliged to consult them. 

It may seem that such responsibility restricts scientific freedom. 
This is not so. Since the Renaissance society has increased the scien- 
tist’s freedom of choice only because it has been willing to assign re- 
sponsibility for consequences to him rather than to representatives of 
other social institutions such as Uie Church. 

Maturation of t)ie child coosisls of developing the abiiity to accept 
freedom and responsibility and to dissolve internal conflicts which 
these create. Maturation of sdence is no different. 

Conflfct bebveen Sciences 

Competition between scientific dlsciplioes for prestige and resource 
is faimliar to all who are acquainted with the institutional aspects of 
science, Competition for prestige is reflected In the scientific snoii- 
beiy to which we have already made reference. This snobbery pre- 
serves an imbalance in the development of disciplines. 

Society reflects the altitudes that prevail within science and hence 
reinforces the scientific caste system This fact is illustrated by an 
examination of the role of sociology in some of the world’s dominant 
scientific powers. I quote from an article I wrote on this subject 
a decade ago [AckoS (1952)]: 

... In the words of the expert on Soviet sdence, Barnngton Moore [1047], 
"Sociology as such does cot exist as a separate discipline m the USSR. Its 
subject matter is inveatigated pnmarity by philosophers and economists.” 
There are no separate departments of socidogy in Soviet universities. One 
sociologist, in s sympathetic presentatioa of contemporaty Soviet sociology, 
asserts that it “merges with history, economics, law, the work of central sta- 
tistical offices of the government, the trade unions and the budget and ptan- 
mng authorities" [Kazakevich, 1944] . . . Sergei Vavilov, President of the 
Soviet Academy of Sciences, refers to the sciences as “physics and mathe- 
matics, psychology, chemistry, biakigy, medieine, gcol^, geography” snd 
“agricultural and the technical aciences" [1947] The titles of the eight sec- 
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tions of the Soviet Academy reflect this omisaion (ie., of the wciol sciences) 
conspicuously. They are : 


1. Phj^ics ami mathematics 

2. Chemistry 

3. Geology ami geography 

4. Biology 

5. Technical (applied) 

6. History and pliilosophy 

7. Economics and law 

8. Literature and language 


From .uch .s lh» S 

cnees, with the possible excepti^ the SUlin pnie in scicace 

Academy of Sciences, has no ^tton ^ u frequently trorse 

la other Western nations, the aituaUon a w letter amna^mq^.^ 

In Ensland, for eaimple.^ in cconeimcs nbsoib most ^ 

partmenta of eociology. There, too, tusi^ ._,.ded for work in the social 
cial Inquiry. Nobel priies have yet to be arrarded for wors 

sciences, (p. 51) .vg egtunated loUl national 

[In the Uruted States] from gs milbon doUars, iMt 

budget for social science iocre.ts^ from . .(j-gjponding budget for 

is, bV . Inelo. .1 a.6. Boring he tlori is, b>- 

:“l1“SnefJr Sn'twiee ns m.eh n. the sneisl *n.e 
hiidget. (p. 64) , 

Dnrins ll.n Inst d««.|o w^ha™ 

Foundation lor the behavioral Ume the Foun- 

lists prevented tlie creation of -uch jjjg 

lialion was established “'““‘“^v.niwd Study ot Uie Behavioral 
establishment of a Center diseiplincs in the seien- 

Scicnecs. However, suppression •' phTrienI scienlisU rctuse to 

Ufic community still place. J7J;-nces and at best consider 
recognize sociology and anthrojm physical seientUU obviously 

p,-ehol.ey m, an ““f^^ris Wependenl nl proems. i« 

believe that progress of their di P 

the liehavioral sciences. tvnes. Th® consists of 

Arguments to the contrary ate o ^ ^ j necessary to pro- 

taking the position that the kmd of aociety 
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an environment in which sricncc can continue to exist, ict alone 
thrive, can be maintained only if the behavioral sciences yield knowl- 
edge which can he used to attain w'orM |jcacc and democracy. Al- 
though I believe that this kind of argument ('.arrics n good deal of 
weight, I prefer n more direct and parochial one, one that asserts that 
accelerated progress of the physical sciences reqmret the development 
of the behavioral sciences. 

When the role of the oh^en’cr in the physical sciences was more 
important than it is today (i.o, licforc elahoratc instrumentation), 
one could argue that errors due to the observer were always present 
and that their riHluction depended on control of tiic obscn’cr. Such 
control, it could he arguori, dc]>rndc<l on incrt*a‘>c<l knowledge that only 
the behavioral sciences couhl provide. 

Such an argument was very fitting when Bessel discovered that the 
obsciwcr introduced significant error into astronomical observations 
through his variable response time to visual stimuli. (Bessel's dis- 
cover)’ is frequently cited as the beginning of the science of psychol- 
og>’.) Although some reduction of (his error was obtained by study of 
the human obsen'cr, the major improvements were achieved by the 
development of instnimcnts which removcil the need for the observer 
in the operations io wldcii be made his largest rontrihution to error. 
This development anil many other similar ones seem to indicate that 
efforts should he made to eliminate the ohsen'er through instrumenta- 
tion rather tlum to improve him 1^* use of knowledge obtained from 
the behavioral sciences. But, although the role of the human being 
in science as an oLserv'cr may lie eventually eliminated, his effect on 
the instruments which replace iiim cannot l>c eliminated. 

For example, those who use electronic computers know that errors 
in computation are all too frequent and (hat most of them are caused 
by human operators. Those errors that are due to the “machine” are 
made generally because some production worker failed to perform In* 
function proper!)'. Somewhere along tlic line the human being is ever 
present as a source of error, and the reduction of this effect can be 
accelerated only by the development of the behavioral sciences. 

Moreover, knowledge of the human being and Ids social envelop is 
neccssarj' to develop better instruments with which to replace him 
For example, rcsearcli is now going on to replace people as tran- 
scribers of information in one form to another that is more suitable 
for computer input. This involves mechanical reading; that is, char- 
acter recognition. To develop such machines it has been ncccssaO' 
to learn a great deal more about such recognition by human beings 
than was previously available. 
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Pcrliars above all vve n-member tliat people are still mpetior 
to lbi”. in many of the (mtetion. r^ninni 
seienec, but even tvhere superior they .till bring «.tb them that very 

pllnS was develo,^ in ^ “"p: ei,ol=r»e bnrl- 

allribulc, and SPiilogram anMjsPi. <lc\ciop»-« » » J 

ing increasing use in the physical sciences. j knowledge 

Failnre to exploit the <~'b '"''"T^irre " ^ P™8- 

developed in diseipUnM Other than OTCS construction of 

,e... Breaking down Ibe /'';'’P'r3jX7lTor”in the past, may 
whicli scema to have occupiwl mu ' , , ^nts of the next gen- 

well be one ol the most peaee and good will 

cration oi acientisU, In science, ns in socaiy, p 
are urgently ncede<l. 

Conflict between Science and Other Social Institutions 

. • !.«= i.Mbn increasing continuously 

The aocial prestige «' been E particularly in the more 

since the Renaissance. This hns wen * . , ,g ^nd questions 

developed eonntrica, in '''■« ^ m be 'TtiiSe authority haa 
over which science is consider^ to scientists h)' religious 

been expanding. The eESed 'X.'. eaX hiatoiy, 

and political institutions, which ' scientists arc wnm«i 

i, virtually removed. There ia * oteur, the eooac- 

away from today; and, ."b'™ ,|,„„„gmg than they once 

qucnces to the scientists invoh jjy^ced" by other institutions 

were. Toiiay’. science n P”XdiltcE types of rcaeareh. 
in the way funds are alloealed to different yp j,, 

Currently there may i” « " reregni.ed that 

tlian in its suppression. lU autn ^ j,^ve increasingly 

manufacturers who are superiority of their prod- 

introduced pjeudo-scientific that science can be used 

ucts. They have ‘^ecs are available for such pur- 

to prove anything and that its sc continue, it 

poses if the proper price is jn science, 

could well undcnninc „ into lop-level management dc- 

Science is increasingly Demg m 
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cision making in public ancJ private institutions. Such developments 
as operations research, management seience, and systems analysis and 
engineering have brouglit science into the domain of important social 
and economic problem solving. Those who use these services arc not 
always aware of the limitations of current technology. Consc<iucnlly 
they frequently use the aid of scientists even where the latter have 
little more to contribute than judgment Because most scientists are 
reasonably intelligent and w-ell educated, their judgment is often 
sought and treated as ate their research skills. Herein lies a d-anger. 
It lies not in the scientist’s use of liis judgment but in his failure to 
emphasize the difference in results and recommendations based on his 
research and those based on his judgment Since his judgment is 
more likely to be in error than is his research, what error of his judg- 
ment is exposed reflects on fcicnce, unless the difterent grounds for 
his results arc kept distinct in the user's mind. 

Because of the extensive use of science in managerial problems (in 
the government, militarj’, and industry) there is a growing fear on 
the part of managers that science is trying to take over their function. 
Nothing could be turtlicr from Uie truth. 

It is true that a large number of repetitive and routine problems of 
management can be reduced by scientists to decision rules which can 
be applied by clerks or computers. This does relieve management of 
some problems, but it does not put them into tbe hands of the scien- 
tist; it relegates them to less skilled personnel or computers. 

This process frees management to devote more of its time to broader 
and more important problems, particularly long-range planning prob- 
lems which arc generally seriously neglected. Here too science has 
been making an increasing contribution, and this, perhaps more than 
anything else, has led to tlic impression that scienee will take over 
management. But this fear is misplaced for two reasons. 

First, most complex management problems cannot yet be com- 
pletely analyzed by science; not all the factors can be handled quan- 
titatively. In almost all broad management problems the solutions 
o^ered by science arc conditional, and the conditions that pertain 
must be established by managerial judgment based on long experience 
with the factors involved. 

Second, the fear assumes that the skills and knowledge required of 
management are fixed and stable, and this is false. The manager of 
today is much better trained in technical matters than was the man- 
ager of yesterday. His knowledge of science and mathematics has 
expanded. It will continue to do so. The manager of tomorrow will 
be quite difTercnt from the manager of today. He will know at least 
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aa much of science and inaihcroatlcs as did the scientist of yesterday. 
A ma}or morement has already started in management education to 
bettor equip future managers in the skills and knowledge that charac- 
terize the scientist. Ilcnre in the domain of roanagement, managers 
will take over science, rather tlian aeieotista take over management. 
These educational developments will bring roanagemetit nnd science 
closer together than Uiey are now and will increase their collabora- 
tion nnd reduce the mutu.al distrust which still charselcrizes many of 
their associations. 


TIIE AESnime IDEAL OF SCIENCE 

To enlarge the range and scope of questions and 
probUms to trhkh srtence con be applied. 

SeicntiGc progress has been defined not only in terms of Die in- 
ercasetl effieicne)* of research but also in terms of the seopc of eciea- 
tiflcaliy derived knowledge. In each age we can witness arguments 
against the possibility of applying science to classes of problems which 
succeeding ages study scieoti/ically as a routine matter. Arguments 
once prevailed against a science of psychology. Tlie ailment still 
rages on the possibility of social science and a science of vaiues. 

Hisloiy* is “on the side of' an expanding domain for science. One 
by one the curtains separating science from various aspects of reality 
have been opened. In the ideal one might suspect timt all probteina 
will be susceptible to scientific study. At least there is no good reason 
to suppose a limit to Urn expansion of the domain of problems to which 
science can be applied. 

This continuous expansion of the scope of science requires both rec- 
reaUonal and inspirational activities on the part of scientists. In so 
far as scientists are human, and they are to a considerable extent, 
they resort to norma! recreational and inspirational activities. They 
play baseball and listen to classical music, watcli television, and ex- 
pose themselves to comic atrips and great paintings. But in addition 
to these there is a ''private" type of recreation and inspiration which 
is unique to science. 

Puzzles and games requiring mental exercise are forms of recreation 
enjoyed by manj’ be.'jides ecionlisfs, but are forms created in the 
main by scienfiste primarily for Uiclr own amusement. It is common- 
piace in research or acndemic instftutions, for example, to find scien- 
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lists relaxing over nen* games or I'sriations of existing games which 
they have invented, Scientists have created many variations of chess 
which fit well into lunch iiours, and the}' have developed unique ways 
of gambling to determine who will pay for the refreshments during a 
coffee break. In gener.al these games arc not ones of pure chance but 
ones in which gtratcgj' is relevant and intelligence Ims a pay-off. 

Humor plays as important a recreational role in science as it does 
in society. For example, I-ewis Carroll (Cljarlcs Dorlg»on), who con- 
tributed so much to children's literature, also made significant con- 
tributions to scientific humor. Artielea along thc«e lines appear regu- 
larly in professional scientific journals. Tliree recent efforts come to 
my mind: an article describing how to l)ecomc a project leader [Ljmn 
(1956)], a mock rc.«carch report of a study of the optimal siic of a 
Committee (Obis (1016)], and the stoiy of Little Red Riding Hood 
told in symijolic logic jRinebarl (I9.>9)|. 

In science itself there arc many sources of in«pirution This is 
manifested, for example, in tlie response to a particularly elegant 
proof or argument by the olwcrvation, “It is beautiful." The appli- 
cation of the word beauty to demonstrations in science is not a loose 
usage; it reflects the recognition of some of the same properties present 
in great art. A proof, for example, may have a structure in which 
beauty is captured. Scientists have long admire*] the great litcraiy 
stylists in science. Great teachers of science have inspired their stu- 
dents by Inducing in them responses to science itself much like the 
response we associate with works of art. Just as we can find beauty 
in natural objects or in commonplace ones that are man-made, we 
can find beauty in the work of scientists. 


CONCLUSION 

As we review the history of mankind, it is clear that there has been 
great progress made relative to economic and scientific ideals Such 
progress is not as apparent relative to ethical, moral, and aesthetic 
ideals. Continuous improvement of the world's standard of living 
and of the domain of knowledge will continue if civilization survives. 
Its survival, however, depends critically on ethical and moral prog- 
ress. The scientist, faced with this situation, cannot just shrug his 
shoulders and wish for the best. He lias a responsibility toward de- 
veloping a science of ethics and morality; that is, a controlled method 
of solving ethical and moral problems. To be sure, responsibility for 
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future morafity is not tlie scientist’s alone, tut perhaps from him can 
come leadersliip in the substitution of brain for brawn in settling con- 
flicts between men and between their societies. 
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AFPENDIX X 

DERIVATION 
OF Min 

EQUATION (16) IN CHAPTER 8' 


To simpli/y flic <ltjcu«ion, without mJurlwR Iw generaJity, we will 
twume that is 5 rvctangutar diMritmCiw Iwtwccn 0 amJ 6; tlial 
«, tlint any value of V, wlicre c 5 1' < h ond a ^ 0, I** equally likely. 
Tills aj»U(npt!oa is not neer^iiy for Ihe aijpimenl which followit; tho 
orgument holds for any other fonn of pO”). 

In onlrr to otitalo the minimiting function d(v), which dcpeiitls on y 
alone, wc first nppnnimatc equation (!5) in Cfiapter 8 liy a finite sum. 
We intraluco o<]ui(li«t»nt mesh pmnti with (ll«tAneM k alwig the 
(y, J*)-o«s in «Jch a way iRu 

o — A' -> cA 


b + ^ ~ nh 

AssJiOHu in Kigun* I.J. tVe further iJc^icnale l?»efnl»l{Ka»l,*<»f Jnlenals 
<4 IcfiRth A liy 

i-e + l.---." (2) 

3/ - O'- i>h, •••»«. a) 

Writing (t5) x* n fnm ct infegrstwn* avrr mc*h Sr.tm'aK we 
*'nu« (VlirtUon M lb* it\*k <4 J>r. Kioil nuMOWMS. 

ta 
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mh n/i 


i; £ r" r Cid(y),Y\pmdudY (4) 
c^-fl /-*•+» •'(i-DK 

= S 2 

i } 

with an obvious definilion of the obbrevialions <7,,. Next, we approx- 
imate each coutributioii G,/ by the product of the area of the square 
between the mesh points involved, and the value of the function under 
the integral taken at the center of the square: 

G„«A’CJd(y.),ryIp(y.|l’». (5) 

Since <Ky) is nn unknQ^vn function, we introduce the unknown ordi- 
nates 

ii. = d(y,), r = c+l. -^n. (G) 

The function d(j/) will be linearly approximated between neighboring 
points consequently, will be considered Joiown when the 

ordinates «. at all me^h points y, have been determined. By (5) and 
(C) wo obtain the followijig approximation of the double integral (4); 

i: z; (7) 

If we introduce the abbrevnition 


( 8 ) 



DERIVATION OF MIN Mlfi(l/) | 449 

or, by (2) nnd (3), 

f,(u,)= X IW, i-t + 1, ■■■,» 

/-»+! 

(9) 

nppitiximtttion (7) tnkfs the form 

i; F.it.,), (10) 

•-e+l 

lYhere the f’,- arc known Xiinctions. Mow' (10) is minimized by minimiz- 
ing each contribution F<(m,) individually Yvith respect to Setting 

*a 0 and defining 

c'(A-,n-;|c(.v,n, (11) 

o.\ 

R'C obtain tlie foiiowing ertuation for the niiuitnuing vulne v. {from (5)]; 

Z c-o.,, lypCuiirrf-o, 1+ i, (12) 

/-*+! 

Tlic problem has been reduced to the solution of n - c Cfiuations each 
of which contains only one unknown. It fan normally be solved by 
hand. Formula (10) tnay be used to evaluate the nrea of reduction 
associated with the solutions of (12). If we arc interested in the expected 
cost for n given value, 1/, we can compute it by tlio approximate formula 


//(>/)= Z C(«.,Jy)pferJ>V), y«i+l, •••,»». (13) 


APPENDIX 


II 


ANALYTICAL DERIVATION 
OF DECISION FUNCTIONS 
FOR A SIMPLE 
INVENTORY MODEL* 


Cansidcr the sinplc invcnfoiy modeJ 

c(A-,}T-xr + i (1) 


where the estimate y of >' has a rcctansubr distribution with mean V 
and range 2IC. In order to fbd the optimal bias, we must minimize the 
following integral; 

f /(y + ftndy. (2) 

•'r-K 

DifTeientiating this expression with respect to ff and setting the dcri^•3livc 
equnl to zero, we obtain 

/lY + K + p. n = /IT - K + e. n o) 


intb our inventory' model 

/(y + a, Y) = 

Therefore, (3) becomes 


/y+ (5 


+ Vj, + 0. 


(4) 


Y 


,+ W+K+i- 


r 


Vr+K-m 

* Theee ore Ute work of Drs. SUt K. Gupta ud Fred Hinsmann. 

450 


vr-x-i-g 
+ Vr-K + e ( 5 ) 
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. r Vi' -K + i- 

’ [ V(r + * 

~ YVY + K + 

■✓('V + «)’ - K’ 

l-> - (V + S)’ - K’ 


and finally, 


Vi’^ + K>«Y + $ 


y. (r.) 

C<,-..i*r,l,ly more otlon so« ioto f.nJine mrolylm ''1’^“'”™ 
ataW^rioHon me.l.od. For o„, inveowo- Problem 

* f7) 

C0‘.,l'()-r.rl( + - 

C(o„lr)p(y-.V>)-[-.>-- + ^] 

Summing over the/*! . 


For optimal m. 




E p(!r..Fr) 

I /.>«-»» 

I £ 


Norrl -VO frrrd -Tal'iSlh^ 


By examining —- 
relationsliipa become clear: 

Fort » 1, . 

S/p(yi. i'/) “ ’(21fc+l)("*'**^ 


( 10 ) 
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For i ~ 2k + I, • • • , m — k: 

^jPiVh Yj) = 
For i = m — k, • • •, fc: 

Sspiyu Yi ) « 


2fc+ 1 


(2fc + l)(»t - k)h^ 

n~i+ I 


(2fc + l)(m - 

With a rectangular diatribwtion of yi, mj know that 


i-t+i i-*+i (2fc + l)(fn ~ 


Then 


/-t+t 


and we can establish the following recursive relationship: 
Forf » 1, •••,2it+ 1: 

D. ■=/),., + (■•+ I -}). 

Solving, , 

D, -l + i + Ci-iKt- J) + Zi 

j-a 

«= (ft + ^) + (i — i)(ft — ^) -j. — ~ — " 

-■■(‘ + i+5 + 5-') = ‘(‘ + 5> 

Substituting into (0), 

«7(2ft + l)(m ~ ft)A=» 


Or, for t = 1, • • 2fc + 1: 


.■(it + D/(2t+l)(>»-W 


(11) 


(13) 

(H) 

(15) 

( 10 ) 

(17) 



ANALmCAt, UEIin-ATIOS OF RECISION Fl'.VCTIONS il 
Tasu: IM. CxjutrtK 
A - 1. K “ 3, f - Hi it - 3. m = 17. w - 20 



4 5 G 7 X 9 to it iZ 13 ll 15 13 17 

(/;• (21 1 

1 

I 

1 3.5 

2 

I I 

2 8 

3 

1 1 1 

3 135 

4 

till 

4 20 

5 

1 1 1 1 1 

5 27 5 

C 

1 1 1 1 1 1 

« 30 

3C + 17 

1 1 1 1 1 1 1 

7 45.5 

8 

t i 1 ( 1 1 t 

7 525 

0 

1 ( 1 t > I > 

7 5!).5 

10 

1 1 1 1 i 1 1 

7 06.5 

11 

1 1 1 1 1 1 1 

7 73.5 

12 

1 1 1 1 t 1 t 

7 80.5 

13 

1 1 1 1 1 1 I 

7 87.5 

m-A U 

1 1 1 1 I i 1 

7 91.5 

2.5 

i 1 > 1 i ( 

0 84 

10 

11111 

5 72 5 

17 

1 1 1 1 

1 CO 

18 

1 l 1 

3 46.5 

10 

1 I 

2 32 

20 


1 IC5 


r, - 0 - l)A 


* (I) *• (2t + l)(pi - k)h* Z 
t (2) - (2fc + l)(m - A)A« Z *'*>• 

To find the recursion relationship for * from 2fc + 1 to « ~ note: 

/ 2i+l\ 

t>3i+t = (2* 4- J) + — ;r“y 


2i® + 2i-* + * + -t + i+ J 
4fc* + 3Jfc + 4. 


( 19 ) 
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Then, for t = 2k + I, ■ • m — fc: 

Di = Oat+i + [i - (2fe + l)l(2k + 1) 

«= 4fc® + 3fc + i + 2* - 4fc* - 2k + t - 2fc - 1 
= i(2fc + !)-{*: + ^). (20) 


"“Vs 


(2k + l)/(2fc 4- l)(m - t)k* 


VI«(2fc + 1) - (fc + |)l/(2fc + l)(fn - k)h 
Or, for » = 2fc + 1, • • •, m — k: 


VA(t-- J>' 


(21) 


To &od D, for < from m — fc to n, wc start with Dm and work back- 
wards: 

D, «n-fc-i. (22) 

For i *s fn — fc, • • •, n: 

Di - D«+i -I- 1 - fc - J (23) 

.'.Oi = n - fc - i - (a - t)(fc + i) + Si 

= n - fc — J - (a - »)(fc + ^) 

(n- 1 -f+l)(a- l + »- 1 + 1) 

2 

= n-fc- §-nfc + ft + - + " 

2 2 2 

= ^ 2i + 2ft - 2ift - 2fc - U 

= 5[(« + *)(" - 0 + 2f - 1 - 2fc(n - i + 1)1 

= il(n + t)(n — * + 1) - (« + 0 + 2» - I - 2fc(n - t + 1 )] 

= 5 l(» + 0(a - » + 1) - (n - f + 1) - 2fc(n - . + 1)]. (24) 



Therefore 
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'' -s/f ft- i 


2(n-t + l)/(2ik+ l)(w-fc)A* 


+ I)(n + » - i - 2k)/(2A + 1)(« - 


\fi(n + i-2t~ I) 
Or, far i «= w — fc, •••, n: 


yh(m-k + i-l 


Asymplolic Values of ^ as A--» 0 
Since Vf - (i - then 


( 20 ) 


Now we solve for the asymptotic value* of asA goes to aero by being 
contlauatiy halved. 

Fort-1, -'..SA + l; 



Then 



( 27 ) 


Fort - 2A+ 1, 


Pi *= 




•150 sciE-vTiric McrtfOD 
Then 

.. A- 

»-0 

I'or I “ m — A-, • • , n; 



The ranRcs of y as A — ♦ Onre as /olioKs: 
For « 2fc + 1, !/< « 2AA + 5 


aa A -4 0, j/ - 2ltA - 2K. (30) 

For * * m — A, yi - wA — JLA — - 


as A -4 0, !/ " mA — lA 

« r + K - K - F. 

(31) 

Thereiorc, summarizing, 



0 < y ^ 2K'; 

a/a’ + to/S 

(32) 

2f^<y< F; 

-4 

(33) 


I 2 

yjF + y' 


F <y<L; a = 


(34) 
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